A small datacenter build and other projects

T

The Spyder

2[H]4U
Joined
Jun 18, 2002
Messages
2,628
*Updated for new readers

I work as the IT Manager for a small, but growing company building custom 3d Image processing and terrain mapping software. Our software runs on custom built portable clusters and storage systems.
In this thread you will find my photo blog of ongoing projects since I started and updated as often as I can. Current projects include building a small warehouse data center, upgrading our multi site network,
and rebuilding our virtualization environment.


To answer some basic questions:
Permission has been granted by my boss to post the following photos.
Most of the gear in cases are from SKB and is customer owned.
We have over 2PB of storage across multiple countries.
Each cluster consists of 20 nodes (Dual Hex Core, 96gb ram, 2x 256gb SSD), 2 Controllers (Dual Hex/96gb ram/SSD), a Datastation (Dual Hex/196gb/SSD/LSI controllers/336tb spindle disk.)
We outgrew 1gb, tried 10gb, but ended up using 40gb Infinband.
They do not make hardware fast enough for us, which keeps things interesting. :)


Pictures from when I started:

Engineering+ Cluster1 server room (temporary, we have 2x 30amp 120v and 2x 20amp 120v power)
juYpsl.jpg


Main server room:
aOyqSl.jpg


My 24kw (2 Ton) AC, largest I can get right now due to other restrictions.)
Fphkjl.jpg


Oh jebus...
wTXabl.jpg


Main servers, storage, ect.
duixKl.jpg


Cable management? Naaaaa....
zTnWml.jpg


It dosen't get any better...
zAw2Wl.jpg


"Engine Room" network closet where the internet comes in...
9HhNol.jpg


At least it's pretty!
dY3FPl.jpg



U7dOzl.jpg

nDKyrl.jpg

Eils8l.jpg

6Leqkl.jpg

PdCNml.jpg

xrc65l.jpg

JKu7El.jpg

nZrXVl.jpg

2OY7dl.jpg

tD9Erl.jpg

4YKYSl.jpg

HNvLVl.jpg

wzSXKl.jpg

qlnnml.jpg

d3x8Ql.jpg

OOl0Bl.jpg

QJp3Kl.jpg

KbXAFl.jpg

WrlSIl.jpg

lRIMIl.jpg

yIFRKl.jpg

oNTcel.jpg

Sk0vkl.jpg

jkUeOl.jpg

uIvBcl.jpg

2SmsDl.jpg

ccU28l.jpg

Id9Rzl.jpg

A5lX9l.jpg

TFkl5l.jpg

ebmsYl.jpg

3CORrl.jpg





SF9VVl.jpg

HFWOdl.jpg

uc8t9l.jpg

RdhIWl.jpg

9pVykl.jpg

LhZsSl.jpg

EMubDl.jpg

6myaCl.jpg

xDgErl.jpg

7gx1rl.jpg

4yCxul.jpg

MLbnTl.jpg

q8tMYl.jpg

O8ymUl.jpg

v72Ucl.jpg

C86i6l.jpg

HrIB8l.jpg

xbsa8l.jpg

DGpB0l.jpg

xKEOtl.jpg

5M6qsl.jpg

fVGHol.jpg

IMJPql.jpg

wkR0dl.jpg

rg1fml.jpg
 
Last edited:
Just FYI a lot of your pictures are repeated.
Looking good though.
 
This is only what I can show, well at least for now :)

We are currently hiring software engineers, if you can't find the name of the company and website based off the pics, I wouldn't hire you :p
Since I am one person for all of this, I am begging my boss for a helper.
 
I have no how one person could handle this. If they don't give you a helper.... I'll do it for free.
 
In 2.5 weeks I did 127 hours? I left at 3:30 today because I was 16 hours over for this week already. Thankfully I have nothing to do with production (software or hardware) and I just maintain the gear. :)
 
The Spyder said:
This is only what I can show, well at least for now :)

We are currently hiring software engineers, if you can't find the name of the company and website based off the pics, I wouldn't hire you :p
Since I am one person for all of this, I am begging my boss for a helper.
Click to expand...

1st Ave, Portland, right? Or are they just one of your suppliers?
 
My god I wish I could get investment like that from our Multi national multi million $ company!

Trying to get them to buy 1 VNXe has taken over 6 months and I am still no further forward $10,000 on a few Apple PCs is easy to get the OK for though.
 
Well, that gives me motivation to keep studying and getting certified, so someday I can be part of an epic project like this!
 
Origin_Unknown said:
what are the green usb dongles handing out the back of a few of the servers??
Click to expand...

Most likely the boot drive maybe?

On my supermicro storage devices, we run FreeNAS for basic storage (NFS) and I use USB drives as the boot drive for them.
 
No kidding I just got done drooling as well.

This is some Sexy work here man. Keep the posts coming. I wish I was building a new data center. I don't think I'll get that option for a while. I'm on my way out the door of this job as of Friday into a much larger company. Not sure what I'll get to do
 
corge said:
Most likely the boot drive maybe?

On my supermicro storage devices, we run FreeNAS for basic storage (NFS) and I use USB drives as the boot drive for them.
Click to expand...

I am guessing software keys. If they were booting off the flash drives they would be inside the servers. Pretty much all modern servers have at least one internal USB port for booting off of.
 
Between being sick and overloaded, I present you with more assorted pictures. Hopefully this weekend I can update the posts and clean them up.

YyZKYl.jpg

ue7EKl.jpg

LPbbql.jpg

kbME9l.jpg

TS2RGl.jpg

22X7el.jpg

PpoIfl.jpg

quWbTl.jpg

LllHfl.jpg

PaDVel.jpg

VbEzbl.jpg

loCqUl.jpg

ukovGl.jpg

Yc3osl.jpg

YUMfal.jpg
 
Last edited:
Nice tools. Those R210's are cute little servers aren't they?

What company makes those rolling racks I keep seeing? I have some mobile PBX equipment that could use a nicer home.
 
Nice dude. One of those cameras could find their way here, my canon 70-200 f2.8L MK II would like that:)
 
I have been a huge Milwaukee fan for years. Until the bastards went to China. I own 20+ pre china Milwaukee tools and every one of them is a tank. I snapped the chuck off my 1/2" Right Angle drill once, they fixed it no charge at the local service center when it was outta warranty. I bought one of their first Made in China v18 drill sets and it has just been a piece. The battery life sucks, the tools are heavy as heck, and they just don't work as well as the old stuff. I bought a M18 impact driver/drill and promptly built two sheds and used it to strip a car. So far it is holding up much better, but I can't help but wondering when it is going to fail.

Now I realize Makita has also moved most its production to china, but it was a 50/50 shot here and I heard good things so I figured it was time to try them. Great little tools. We have the 18v set in our assembly area and the guys love it.My first drill was a Makita 9.6v, I still have it somewhere. Still works too :).
 
WOW! I don't think I can "technically" look at these pictures on my work computer since they are "technically" considered porn. Seriously though, please keep updating these pics. My jealously grows stronger with each one. :D
 
Brb, gonna go play on my 144 GB Dual Intel Xeon X5680 Server.. I got some number crunching to do. :D

Very nice photos BTW.
 
The Spyder said:
My 24kw (2 Ton) AC, largest I can get right now due to other restrictions.)
Fphkjl.jpg
Click to expand...

I think you meant to say 24k btu (if that is the classic 26 movin cool), which is 2ton. 2ton is about 7KW :)
 
Haha yes! This would be a 7 Ton unit. I am adding another 12k BTU portable to our R+D lab. These older x5650's really put out some heat!
 
The Spyder said:
Haha yes! This would be a 7 Ton unit. I am adding another 12k BTU portable to our R+D lab. These older x5650's really put out some heat!
Click to expand...

No, it's not a 7 ton, it's a 2 ton. Kilowatts and Tons are very different units of measurement.
 
Who is the manufacturer of those rolling racks? I could use a few of those for some mobile PBX gear.
 
You must log in or register to reply here.
Back
Top