-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Conversation
"name": "Julia", | ||
"image": "julia", | ||
"extensions": ["julialang.language-julia"], | ||
"postCreateCommand": "julia --project=. -e 'using Pkg; if (isfile(joinpath(pwd(),\"Manifest.toml\")) || isfile(joinpath(pwd(),\"JuliaManifest.toml\"))) && (isfile(joinpath(pwd(),\"Project.toml\")) || isfile(joinpath(pwd(),\"JuliaProject.toml\"))); Pkg.instantiate(); end'" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Generally we're trying to keep the postCreateCommand as something developers can use as needed. These files are dropped into an existing project. Is this something that every developer would want to do with source files that already exist?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think this makes sense for everyone. It essentially just makes sure all the packages that are specified as a dependency in the root Julia configuration file in the repo are actually installed on the machine and available right away.
I guess in theory we could also try to put this into a docker script file, but then we would lose the ability to use the pre existing julia image files and instead would end up using building a new image for every single repo, which seems a lot less nice from a performance point of view?
We've been doing a very similar thing in the existing Julia mybinder.org integration for a long time and it seems to be what users want.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, there's a bit of a balance between keeping it simple and getting people started that varies between ecosystems.
Generally we've been adding Dockerfiles to help people get started on adding their own contents to the image. (For example, this Dockerfile). That's not strictly required though - it depends on how likely someone is to want to do something like this.
One thing I did notice, however, is git
is not present in the base Julia image. That's a pretty common dependency. There is a script that can be used that pulls in common dependencies including git. You are right that it can increase the time it takes to get the container up, however. You can default to not installing zsh and not upgrading packages to help there a bit. e.g.
# This Dockerfile adds a non-root user with sudo access. Update the “remoteUser” property in
# devcontainer.json to use it. More info: https://aka.ms/vscode-remote/containers/non-root-user.
ARG USERNAME=vscode
ARG USER_UID=1000
ARG USER_GID=$USER_UID
# Options for common setup script
ARG INSTALL_ZSH="false"
ARG UPGRADE_PACKAGES="false"
ARG COMMON_SCRIPT_SOURCE="https://github.com/microsoft/vscode-dev-containers/master/script-library/common-debian.sh"
ARG COMMON_SCRIPT_SHA="dev-mode"
# Install needed packages and setup non-root user. Use a separate RUN statement to add your own dependencies.
RUN apt-get update \
&& export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install --no-install-recommends curl ca-certificates 2>&1 \
&& curl -sSL ${COMMON_SCRIPT_SOURCE} -o /tmp/common-setup.sh \
&& ([ "${COMMON_SCRIPT_SHA}" = "dev-mode" ] || (echo "${COMMON_SCRIPT_SHA} */tmp/common-setup.sh" | sha256sum -c -)) \
&& /bin/bash /tmp/common-setup.sh "${INSTALL_ZSH}" "${USERNAME}" "${USER_UID}" "${USER_GID}" "${UPGRADE_PACKAGES}" \
&& rm /tmp/common-setup.sh \
&& apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/*
The package upgrade is mainly something we have for scenarios where a downstream image is actually generated so security patches are picked up before its published.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, git
is a good point! I'll try to figure out whether there is some pre-built image for julia that includes git
, that would give us the fastest story, right? If not, I'll look into the custom dockerfile.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Or would it make sense that we (the Julia extension for VS Code) publish our own julia-devcontainer
image for this scenario? That way we could include anything we think is useful, but avoid the extra time to build a new docker image every time a user instantiates a new devcontainer?
I guess this is a scenario where microsoft/vscode-remote-release#3441 would also help: you (the codespace implementation) could actually cache the docker image that gets build by a particular devcontainer definition that is in the central repo and then reuse that docker image for any user instance that references it. mybinder.org has a pretty neat design where various docker images get cached at various points, there might be some good ideas there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Publishing your own image makes a ton of sense since that gives you control over when updates need to happen to the image. There's a number of other examples of this in the repository - e.g. Puppet, Salesforce, Azure Functions.
The thing about microsoft/vscode-remote-release#3441 that is not covered by the image alone is the various devcontainer.json
settings. In some cases these can be more involved, so we've considered the idea of a "inheritance" for it where you can start from a definition and then override, but that doesn't diminish the value of a published image.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, the custom image works great. I just need to figure out how we can update it regularly and make those kinds of aspects a bit more stable, and then I'll update this PR here. Thanks for all the help and guidance!
Re: Test infrastructure, there's a job that runs through and fires "test-project/test.sh" on a regular basis after spinning up the container. If the file has a non-zero exit code and we determine it's not related to an extension change, we'd raise an issue here and CC the owner of the definition that failed. So, you can put anything in there you think is needed to smoke test the Julia image in this context. That help? |
Alright, I just added a test that runs the Julia binary with the Let me know if you think I should change anything else with this PR, I think from my end this is ready. |
Nice! Looks like we posted at the same time. One comment above: #450 (comment) |
@davidanthoff Nice work! Really cool feature 😄 I noticed that some changes were requested by @Chuxel. I started on the requested changes here. It is still a work in progress. Once I'm satisfied what would be the best way to contribute? Should I ask @davidanthoff to merge my changes or should I make a PR to microsoft/vscode-dev-containers directly? |
@ThomasHagebols very cool the progress you made there! The thing where I got stuck is that I thought it would probably be best if we hosted our own Julia VS Code docker image somewhere instead of adding a full I think we should probably host these images on the just announced Github Container Registry? I'll try to figure out how that actually works, and then I guess we would use the |
Agree, having Where would you suggest to host the the Dockerfile / image? https://github.com/julia-vscode/julia-vscode? Let me know if I can contribute in any way |
@ThomasHagebols, yes would be great if you could help with this! I've set up all the infrastructure to host our own images etc. If you could update the The The images are then hosted at https://github.com/orgs/julia-vscode/packages/container/julia-devcontainer/. I still need to fix which of the versions that we host it considers the "latest", other than that it should all work. https://github.com/davidanthoff/StringBuilders.jl/blob/master/.devcontainer/devcontainer.json has an example of how this can be used. Essentially it is now Once we have the |
Cool, I can pick it up end of week. I will update the Dockerfile and update the accompanying readme |
So, I think this is ready to be merged, thanks to @ThomasHagebols' help! The repo that builds our docker images is at https://github.com/julia-vscode/julia-devcontainer, and the images themselves are published at https://github.com/orgs/julia-vscode/packages/container/package/julia-devcontainer. Once things are merged here, it would automatically show up in some new release of VS Code, right? Or the remote extension? |
LGTM! One note is that we're discussing switching the default user from "root" to the "vscode" user, but given that's already in the image, I can add that to the PR I'll submit on the topic. I'll also make a couple of tweaks to the readme and add a comment pointing to the source for the image just so people have it. That said, yes, the next release we do out of this repository will have the definition in it. |
Fantastic, just discovered it in the shipping build :) |
@davidanthoff Would you be willing to consider a devcontainer with conda + jupyter + the python datascience extension merged in as well? |
Ok, now that I have tried it, I think it is not yet ideal: it now adds a Another, unrelated question: @pfitzseb had a better idea how we could a) simplify the And another question: I've seen @jlperla I think that should probably be a different devcontainer, I'd like to keep the default Julia one simple and lean. But having a "all things datascience" devcontainer template in general seems like a good idea. |
100% agree. After this gets solid we can see if merging in the minimal jupyter stuff necessary into a |
Are there instructions on how to use this? |
I don't understand the testing infrastructure, so I just left the test script from the template in here. Should I delete that, or do something else with it?