Even more compilers for GCC explorer

As promised in the last post I’ve added GCC 4.9.2 to GCC explorer. Also new is clang 3.5.1 and experimental builds of clang 3.6, and clang 3.7 – all for x86.

I meant to do a GCC 5.0 snapshot but I haven’t gotten around to that yet; hopefully later in the week I’ll have time.

Filed under: Coding
Posted at 23:50:00 GMT on 18th February 2015.

Announcing new compilers for GCC Explorer

I’m very pleased to announce that GCC Explorer now supports some new compilers, and new architectures:

It’s been a long process to get here: keeping all the old Ubuntu 12.04 compilers but supporting these newer ones on Ubuntu 14.04 was tricky. I ended up using a 12.04 Docker image (after much pain) and then adding support to passing through to other GCC explorer instances.

I hope to add more compilers soon. Next on my hitlist is GCC 4.9.2 and a GCC 5.0 snapshot. I’ll update here when I do.

Filed under: Coding
Posted at 05:00:00 GMT on 9th February 2015.

Experiences with Docker and GCC Explorer

GCC Explorer has long ran on Amazon EC2 Instances which have been marvellous.

I’ve been looking to upgrade from the Ubuntu 12.04 images I was using to Ubuntu 14.04. But I had a problem1: most of the compilers I was supporting were only built for 12.04. I was using some extra PPA repositories to get them, and while some compilers had been updated to 14.04, many were now on new versions. As I wanted to support the old versions still, I had a bit of a problem.

Enter Docker - a container system for OS images and data, like a very lightweight VM. Yes, this means running a VM (Docker) on a VM (Amazon EC2 instance) on a real machine…but it’s turtles all the way down anyway.

Docker has made it possible to build a 12.04 image including all the compilers I currently use, and then run that image on a 14.04 box. My next plan is to get a 14.04 image to run too and then use some RPCs between the containers to present both 12.04 and 14.04 compilers to the web front-end. But anyway, thoughts on Docker.

Docker’s great. But there are some things that took me a while to “get”:

Containers vs Images

In Docker, a non-running “VM” is an image. When an image runs, it runs in a container. Both can be tagged and named, and both can be referred to by tag, name or UID (a big hash value a la git).

I conflated these two ideas and it took me a while to understand them. Partly this is because you can “run” a container from an image, and you can “commit” a container to make a new image. But, the image is the non-running, shippable “VM image”, and the container is like a VM instance.

You can list containers with docker ps. This will show any containers that have running processes. But even if this list is empty, there may be containers on your system. docker ps -a shows them all.

Images are shown with docker images. There are also a docker images -a which shows even intermediate images – docker layers diffs between images so there’s a ton of in-between steps.

I think what confused me is that one can run a command in docker and specify an image to start from.

$ sudo docker run ubuntu:12.04 \
    bash -c 'echo "hello" > /world'

Woo! We created a little message in /world in our ubuntu:12.04 image. Let’s try reading from it:

$ sudo docker run ubuntu:12.04 bash -c 'cat /world'
cat: /world: No such file or directory

Oh! what happened here? I’m sure we created a file, but where has it gone?

So…I’ve conflated images and containers again. If you run a command from an image, you implicitly create a new container instance, and run your command in that. Any changes are now reflected in the container, but not the original image. When I can the cat, I created a new fresh container from the Ubuntu image, and of course the file isn’t there. So where did it go?

$ sudo docker ps
CONTAINER ID   IMAGE          COMMAND

…well, not there. Maybe it’s not running?

CONTAINER ID   IMAGE          COMMAND
20f2d1dc8f23   ubuntu:12.04   bash -c 'cat /world'
8d87f95f0ae9   ubuntu:12.04   "bash -c 'echo "hell

Aha! I can see now both new containers. If I now try running my cat command with the container ID instead:

$ sudo docker run 8d87f95f0ae9 bash -c 'cat /world'
Unable to find image '8d87f95f0ae9' locally
Pulling repository 8d87f95f0ae9
2015/02/02 08:41:07 HTTP code: 404

Hmm, well that didn’t work. It seems to have tried to find an image named 8d87f95f0ae9 in the docker repo. You can’t run a command in an existing container. Some Googling reveals the presence of a docker exec feature, but that’s not made it to Ubuntu 14.04’s version of docker yet.

The only way to modify an existing container on is to docker commit it into a new image, and then run on that.

$ sudo docker commit 8d87f95f0ae9 test
d317cbff7cff1c554760424651edf141addf2a3b5c4
$ sudo docker run test bash -c 'cat /world'
hello

Yay!

Dockerfiles

All that committing and running seems like a right pain, right? Luckily that’s not too much of an issue as the whole process of setting up your own image is made fairly simple by the docker build stuff, which takes a Dockerfile script of commands and automates the whole process.

A snippet from one of the GCC Explorer Dockerfiles:

FROM ubuntu:12.04
MAINTAINER Matt Godbolt <matt@godbolt.org>

RUN mkdir -p /opt
RUN useradd gcc-user && mkdir /home/gcc-user \
    && chown gcc-user /home/gcc-user
RUN apt-get -y update \
    && apt-get install -y python-software-properties
RUN add-apt-repository -y ppa:chris-lea/node.js \
    && add-apt-repository -y ppa:ubuntu-toolchain-r/test

RUN apt-get -y update && apt-get install -y \
    curl \
    s3cmd \
    make \
    nodejs

After each RUN stanza, an intermediate image is created. The docker build agent tries to cache build steps, so if you’re careful you can get a pretty fast build if only the latter steps change.

The rest of GCC Explorer’s Dockerfiles are on github if you want to take a look.


  1. More than one…my URL format was terrible and the pathname of the compiler was specified…of course then upgrading OS would inevitably put a different compiler at /usr/bin/g++. I’ve now fixed this… 

Filed under: Coding
Posted at 23:00:00 GMT on 2nd February 2015.

GCC Explorer's top 10 compilers

I’ve had a post-it note on my computer for the over six months to upgrade the GCC Explorer cluster to Ubuntu 14.04 from its current 12.04 setup. One issue I have is that newer Clang versions only have pre-canned binaries for 14.04, and rather than have to build my own, I’d prefer to use those binaries.

That being said; the reverse is true for older compilers like GCC 4.4.

I wondered if I could drop support for GCC 4.4, and in short, I don’t think I can! I turned to the very minimal Google Analytics I have for GCC Explorer, and the top ten compilers (in nearly 2 million compiles) are:

  1. clang++ 3.0.6 (the default when you visit the site) – 37%
  2. g++ 4.8 – 15%
  3. g++ 4.9 – 13%
  4. clang++ 3.4.1 – 6%
  5. g++ 4.9 concepts branch – 4%
  6. clang++ 3.3 – 4%
  7. g++ 4.7 – 4%
  8. Intel’s compiler 13.0.1 – 3%
  9. ARM g++ 4.6 – 2%
  10. g++- 4.4 – 2%

So, venerable old g++ 4.4 is still getting 2% of the hits to the site (40,000 compiles).

I suppose I’ll have to bite the bullet and get builds of gcc4.4 for Ubuntu 14.04. I also really ought to update the Intel compiler too!

Filed under: Coding
Posted at 19:00:00 GMT on 16th December 2014.

About Matt Godbolt

Matt Godbolt is a developer working on cool secret stuff for DRW, a fabulous trading company.