CERN Computing Centre (and mouse farm) – Computerphile
Articles,  Blog

CERN Computing Centre (and mouse farm) – Computerphile


Okay, then. We’re in and this is the computing center at cern [I] mean really this is the brain of the large Hadron collider Now it’s really loud [in] here. It’s not a place where I can easily do interviews or give you a live commentary so I’m recording this afterwards, but I just wanted you [to] see what it looks like and hear what it sounds like When you walk [through] these kind of little corridors here I can’t tell [you] how hot it is the hot air is just blasting you from both sides it’s like being in a little oven and Of course, it’s so loud. I mean you can hardly hear yourself think Now this is really the heart of what cern calls the grid which is a network of computer systems and centers around the world That’s used to process the insane amount of data produced by the Large Hadron Collider the LhC Produces about 600 million particle collisions our second and although they don’t keep everything. I mean that would be crazy They just keep a tiny 100,000 snapshots per second that’s still a fair [amount] of viously and that means they’re storing about where they’re storing over a Thousand megabytes per second and it makes its [way] here to the data center now the data doesn’t just need storing it needs Processing and that’s where the grid comes in This center is the start of it all and they call it tier zero. It’s the tier zero center Then there are tier one, you get tier twos, and you get tier threes, and they’re scattered all over the world Together they allow thousands of physicists to access all this information and more Importantly crunch all the data process [it] make sense of it all and I mean just the tier 0 Center here alone does a million jobs a day and it can transfer up to 10 gigabytes of data Every second and I mean these are just numbers obviously, but when you think about it They’re pretty high numbers Now as you can [see] here some of these banks some of these racks of computers are kept in kind of these glass enclosures I Didn’t actually ask about that, but I’m assuming it’s all sealed so that it’s more easy to to keep air-conditioned and to keep cool Calling this place is just so important and in fact here you see these big blue towers [I] mean, that’s part of the air conditioning system. [I’m] pretty sure they told me if the cooling fails everything gets switched off very quickly Otherwise they have bit of a meltdown I do love that you sort of have these little keyboards and computers on the end of the rex [too] I mean, I mean it’s obvious that your wood, but it’s somehow amuses me I’m not sure why I mean I sometimes think the cables behind my computer and television are complicated when I see this place I feel like my head’s going to explode Now we’ve come upstairs the [Center’s] actually spread over two floors and here We see the cartridges or tapes that a lot of the information stored on One of the storage units here for the tapes was actually being worked on while I was there so the door was open and I Could have a look inside Now often when the information on these tapes is accessed. There’s kind of this robotic, sort of a device type thing that scurries around the racks and Pulls out the tapes [and] whenever I wasn’t filming it would go whizzing around doing all this cool stuff But as soon as I started filming it all kind of stopped and I had to wait for ages But I was determined to show you one in action and eventually eventually I’ve managed to get this You get the idea okay now. We’ve come up to a little sort of room where you can see over the center This is the [ground-floor] part of the center again You can see how high the ceiling is that’s actually not a good thing modern data centers actually have low ceilings I think it’s because it’s easier to cool the rooms if the rooms are smaller But this was built at a time when well computers were a lot bigger it’s actually in a lot of ways the building in the rooms quite outdated for the purpose, but They’re still using it [I] think it’s certain they prefer spending their money on a new kit rather than new buildings and They’re still doing a pretty good job and here’s one last thing. I want to show you it’s actually outside the computer center They call this the soon animal shelter for computer mice It’s bit of a joke But it’s where the it’s where the computer mice go to retire when they’re then no longer in use Is mice the plural [where] [the] mouse’s must be mice? Anyway there it is. It’s a bit of fun now [as] I said it was remarkably loud in the center, so it wasn’t really easy to do interviews But I did do interviews [with] people that work here at the computer Centre You’re going to be [able] to see them [very] soon here [on] computer file Sean who I work with is going to be having all that going through all adding it together soon So stay tuned for that also while we were at cern We made a music video for the sixty symbols channel if you haven’t seen that yet Go and have a look it was a bit of fun And I also recorded something about the corridor where the world wide web was developed and that video is already on computer file So you can go and check that out, too? There’s so much to look at thanks for watching this and we hope to see you soon for more videos

100 Comments

  • Melthornal

    It is strictly nonstandard, ie incorrect English, to say "computer mice". It doesn't matter if you care or not, you aren't the one who gets to decide.

  • Melthornal

    You are very much wrong. Mouse is *not* an acronym. The acronym was invented afterwards as a gimmick. Second, OED only lists 'mice' as "acceptable" because it was the first listed plural of computer mouse. The issue you're running into is that everyone who has any understanding of how English works has recognized that "computer mice" is grammatically incorrect and have changed to "computer mouses." If you were to spend the time to actually read the OED entry, maybe you would understand this.

  • Gai Ikari

    cool, so you finally agree that the OED says Mice is acceptable. So between that and common usage my point is made. Mice is correct and acceptable. Society lead and the OED followed, you stragglers who are so OCD about a back set of rules and compliance will get there in time or just die out and the argument will be forgot and won by default.

    Interesting discussions, bye.

  • Melthornal

    It doesn't list it as actually acceptable, it lists it as an abandoned error in history. There is a difference. OED lists philome, for fucks sake. When was the last time you ever saw philome spelled that way? Never? Oh, maybe you read Shakespeare once? Because I'm pretty sure Shakespeare was the only person to ever spell it that way. Everyone else spells "film."

  • TechLaboratories

    That's what I saw too – All of the servers faced inward to the sealed cold aisle, which they drew their intake air from. While it decreases the amount of reserve cold air in a power failure, it also makes the data center bearable to work in (without a coat…) as opposed to the hot aisle which uses the outside space of the data center for cold air resevoir and pairs all of the servers back-to-back, creating rows of hot.

  • Guillermo Mendez-Kestler

    I agree it does look like a hot cold aisle setup.

    Without knowing the full setup or viewing the interviews, I'm assuming that CERN has some kind of data tiering going on. Maybe the same kind referenced in the video. Otherwise those tapes would be really slow to process roughly a Gig/sec.

  • afhdfh

    I now. 🙂 However, I'm sure that this is only true for conventional methods. We'll probably figure something out eventually. Like quantum computing (though this won't be commercially usable unfortunately)…

  • Dustin Strickland

    Unless quantum computing advances, we're basically stuck right where we're at…. The transistors that we use today can't get any smaller. I also vaguely recall hearing something about using lasers to make a material temporarily conductive, and how that sort of technology could be used to make processors with performance measured in petaflops

  • discflickerDotcom

    next time, please show what it looks like undeneath the raised flooring… underneath those white tiles; wires, cables, lines, etc.look like a nightmare, so every line is supposed to be labeled to track what gets plugged into what.

    the air-conditioning is all blown into this under-the-floor space and flows up into all of the machines seated on the raised-flooor from there.

    i've spent many an hour crawling underneath the raised floor of live computer centers
    untangling unlabeled messes!

  • tfsupp "Technical Facilities Support"

    It is using Hot & Cold isle system with cold isle containment & is the glassed in area. hot isles can get very hot 40c+.
    This is common type of cooling for server farm. There are many types of hot cold isle some with under floor deliver & higher ceilings (Still built like this depending on requirements like fresh air cooling etc) or can be low ceilings & no under floor delivery. There are dozens of combinations & depend on size, location, weather, computing purpose or your particular philosophy.

  • tfsupp "Technical Facilities Support"

    It is using Hot & Cold isle system with cold isle containment & is the glassed in area. hot isles can get very hot 40c+.
    This is common type of cooling for server farm. There are many types of hot cold isle some with under floor deliver & higher ceilings (Still built like this depending on requirements like fresh air cooling etc) or can be low ceilings & no under floor delivery. There are dozens of combinations & depend on size, location, weather, computing purpose or your particular philosophy

  • Jerry Lifsey

    IF Moore's Law continues at computing power doubling every two years (or the same computing power halves in size), this room will be 1/1024th the size in 30 years with the same computing power.

  • Stuntddude

    Yeah, and if you haven't noticed, the room is HUGE. Moore's law works to exclusively either halve the size, double the power, or halve the cost every two years (given, for each, that the other two factors stay constant). Consider, also, that the cost of this whole installation must be insane. And finally, 2^15 is 32768, not 1024. 1024 is 2^10.

  • dzjad

    So the front of the computers are enclosed to provide probably cold air–and the backs are exposed as the outflow of hot air. So colder air is provided only to the exact spot it needs to be (the entrance to the system needing cooling).

  • gtamateur

    Not really… no in our current form of transistor. The law of Moore is near his end except is cubit pc come and save the day

  • afhdfh

    🙂 Can't they manipulate single atoms now? I'm pretty sure that people are smart enough to find a way around Moore's law… 😉

  • gtamateur

    Yeah, but you can't go under a certain limits of atom because Electron can pass trought the transistor's gate even technically in classical physic they couldn't

  • D. Tysen

    Well the strict Moore's law says that transistors will get exponentially cheaper, which means they have to get smaller to fit into a consumer device. A transistor needs to be about 50-100 atoms across so that current can't just bypass the semiconductor, and researchers are already building prototypes about that size. Even if that weren't true, if the trend continues for 10 years we'd be making parts out of individual atoms, and you can't get smaller than that.

  • D. Tysen

    A generalised version of Moore's law would just say that computing power over cost increases exponentially, and that might be true but it would require having a totally different kind of computing from what we have now.

  • AnarchyEngineer

    When I went there about 3 years ago as part of a school physics trip, saying that the server room is a warehouse is an understatement. Heck, it has a transformer outside the size of a small house to power this. Very impressive stuff!

  • Yandy

    Tapes are still the cheapest and most reliable backup storage. Infinitely cheaper and still functional for what they need it for.

  • Thrust vectoring

    Not only LHC, CERN is not just LHC, there are lots of experiments – like ISOLDE, nTOF, SHINE, COMPASS, CLIC, CAST, ACE, AEGIS, ALPHA, ASACUSA, ATRAP, CLOUD, AMS, AWAKE, DIRAC, MODEAL, OSQAR, TOTEM, UA9 and NA62.

  • Mutrify Gaming

    Yes. There is a limit.

    To put it simply computers are made of a lot of transistors connected with "wires".(And of course insulators that stop electricity from going the "wrong" way, and a bunch of other stuff). You need at least X atoms to make a transistor, or a "wire", where X > 1(a lot bigger). You can't make a conducting wire or transistor that's "half" an atom across.

    If that makes sense.(Just trying to explain, the picture is a lot more complex in the real world.)

  • Dawid Stasiak

    If there is, physics has probably got it already figured out, there is a technological limitation more than theoretical 😉

  • Dawid Stasiak

    That's either some next level physics or you're trolling me, i tried to research that on the internet but do not understand it 🙁 but then again I'm in my second year of college doing physics where we still learn about newton's laws of gravitation -.- and i know quite a lot about modern physics, hence why I want to do theoretical physics at uni ^^

  • 666qwertz

    Not really, quantum computers can only do certain (mathematical computation) jobs a lot faster. It wouldnt be much useful for most tasks you do in your daily life using a computer.

  • LimitedWard

    That's only because scientist have not discovered the other uses of quantum computing. Keep in mind, this technology is still in it's infancy. When computers were first invented, we only thought they would be useful for simple arithmetic or tabulation problems for large companies. We never knew the direction computers would take just as we will not know where quantum computing will take us.

  • Morbitron

    That would not do a thing because you new to reinvent the way you process information and I don't think of passing from a binary system (base 2) to a base 16. To now, the only theoric ways of using qBits told us that it is way better a doing parallel calculation instead of sequential.

  • 99codemister

    The language is awesome, the JRE is not. Also, since Notch left the project there was plenty of optimization done. Also, it's challenging to optimize graphics in Java since you can't take advantage of a gpu.

  • Joshua Brockschmidt

    I had no clue what I was looking at when I started the video, but I sensed I was somehow getting a glimpse of heaven.

  • acadianalien

    Everytime I see amazing feat like this, I just want to show it to all those conspiracy theorists who think that we can`t build the Egyptian pyramids with modern technology.

  • Bo Duholm

    I think the blue ducts are blowing Cold air into the enclosures you see through the floor. The servers then suck in the Cold air from the front and blow it out the back. That way you don't have to cool the entire hall, but only the small corridors between the servers and extract the heat from the ceiling of the hall.

  • riskinhos

    CERN's total annual electricity consumption is 1TWh -1,000,000 MWh or 1,000,000,000 kWh:
    40 to 45% is for the LHC (including 12 to 14% for cryogenics and 7 to 9% for cooling and ventilation)
    10 to 12% is for the LHC experiments (ATLAS, CMS, ALICE and LHCb)
    28 to 32% is for the SPS (including 6 to 7% for the North Area experiments)
    2 to 3% is for the PS + Booster + Linac
    5 to 6% is for the Computer Centre (B513)
    7 to 9% is for offices and restaurants, etc.

    CERN is supplied by RTE
    Réseau de Transport d'Électricité (Electricity Transmission Network) is responsible for the operation, maintenance and development of the French high-voltage transmission system.
    RTE is a wholly owned subsidiary of the partially public-owned French generator Électricité de France (EdF)

    EdF:
    nuclear: 84.7%;
    renewable energy: 8.3% (among which 4.6% from hydroelectric plants);
    gas: 2.7%;
    charcoal: 2.7%;
    fuel: 1.2%;
    other: 0.4%.

    so, CERN power comes from nuclear. far from clean. good luck managing the radioactive waste. shame on them. with all that money they could have gone renewable.

  • Alex McMahon

    The consoles at the end are hilarious because it's almost like a simple desktop to more computer than you can imagine. So little to control so much.

  • Capt-Morgan

    In laymen's terms, can someone please explain the point of crashing particles into one another? I understand what CERN has been able to produce in the modern era, but how do you make the jump from particle acceleration, to something like creating the Internet, or other?

  • Capt-Morgan

    3:48 is cool, but nothing epic. Some businesses operate those machines. I would like to learn about their backup method though – imagine the off-site backup of the initial backup? Or maybe they use incremental backup? heh

  • cgdermot

    Why use tapes? would SSD's not be better? I guess they might be reliable but waiting for the selector arm, then the load time. I don't get it.

  • Ryan Mitchell

    "They're just numbers, but when you think about it, they're pretty high numbers"
    To make the obvious comment, I can't help but be reminded of the room filling mainframes of the 50's and 60's.
    Except these are more modular, and fill up entire warehouses instead.
    I was gonna say, even 10GB/s seems kinda low for today's standards, especially considering how many machines where working on it, but then I noticed this video was from 2013.
    I just wanna show someone in the future 1:52–2:06 and watch them piss their pants.

  • Sander Deryckere

    The sealing is also good to keep dust out. You can blow filtered air into the sealed corridor, and it will blow dust and heat out through the computers towards the unsealed corridors. (and the building itself also happens as a second sealing)

Leave a Reply

Your email address will not be published. Required fields are marked *