Very cool hardware project

Discussion in 'Hardware' started by ffreeloader, Jun 11, 2008.

  1. ffreeloader

    ffreeloader Terabyte Poster

    3,661
    106
    167
    Here's a home-made hardware project that has 24 cpu cores and 48 gigs of ram running Fedora Core 8 and Blender (i.e used to create Big Buck Bunny video). It's used to do 3D rendering. It's fast enough that 3D rendering projects that used to run overnight on a PowerMac with 8 cores and 16 gigs of ram now complete in 10-12 minutes. It only uses a total of 400 watts of power and costs no more than a high level Windows gaming machine to build.


    http://helmer.sfe.se/
     
    Certifications: MCSE, MCDBA, CCNA, A+
    WIP: LPIC 1
  2. Phoenix
    Honorary Member

    Phoenix 53656e696f7220 4d6f64

    5,749
    200
    246
    Awesome set up
    and not even that open source orientated, great post freddy :)


    Now imagine your Seti score after a week or two on that bad boy!
    I've been toying with a similar setup for an ESX/Hyper-V farm, this looks like it might just do the trick! and the Ikea case is an awesome fit :)

    I love it
    I want one
    the cool thing is you could grow that sort of farm over time!
    add a system or two as funds permit :)

    shame he didnt give a final price

    Freddy are you aware if they are just using out of the box fedora? does the Dr Queue handle all the distribution? no complicated stuff like Beowulf (Does anyone remember the Beowulf cluster of shuttle PCs?)

    Nice find mate :)

    Edit: Hemmler 2 looks like a nice concept ;)
     
    Certifications: MCSE, MCITP, VCP
    WIP: > 0
  3. Mathematix

    Mathematix Megabyte Poster

    969
    35
    74
    He'll need an upgrade within the next month or two. :p
     
    Certifications: BSc(Hons) Comp Sci, BCS Award of Merit
    WIP: Not doing certs. Computer geek.
  4. ffreeloader

    ffreeloader Terabyte Poster

    3,661
    106
    167
    As far as I can see it's an OOB Fedora install. For that matter I think you could probably use any one of the major distros to accomplish the same thing. He most likely used Fedora just because he's familiar with it.

    I'm not familiar with with the Dr. Queue software at all. It's actually the first time I've ever heard of it. As far as the cost goes I read somewhere that it was about $3500 US to complete it.

    edit:

    More on Dr Queue. It's a GPL project that's included in the Debian repositories and will work with Blender, Maya, and several other rendering software packages. It has a gui package that goes with it and you can start, stop, reprioritize jobs, restart specific frames, etc.... It sounds as if it's pretty powerful. It's basically just a bash script with a gui available for it.
     
    Certifications: MCSE, MCDBA, CCNA, A+
    WIP: LPIC 1
  5. Phoenix
    Honorary Member

    Phoenix 53656e696f7220 4d6f64

    5,749
    200
    246
    damn that really is the cost of a retail high end gaming system (although enthusiasts can put one together far cheaper) still thats not much for a render farm of that magnitude

    the Pixar CFO should take note :)
     
    Certifications: MCSE, MCITP, VCP
    WIP: > 0
  6. ffreeloader

    ffreeloader Terabyte Poster

    3,661
    106
    167
    Yup. It's the low-cost, not cheap, way to go.

    BTW, if you don't think this thread was about open source try figuring out what the costs of this project would be using Windows and a proprietary 3D rendering product. They all charge by the core for licensing, so the real reason this project has the build price associated with that it does is because of open source.
     
    Certifications: MCSE, MCDBA, CCNA, A+
    WIP: LPIC 1
  7. Phoenix
    Honorary Member

    Phoenix 53656e696f7220 4d6f64

    5,749
    200
    246
    I was ignoring the blender reference :)
    I get what your saying, but not being a animator i was admiring the hardware more than what it was used for ;)

    I'd have to say that this probably was a CHEAP way of doing it not just a low cost way of doing it
    buying consumer hardware for an enterprise rendering farm just would not be possible for most corporates, (thats cheap, not low cost)

    using blender and linux most certainly would though, (that bits low cost not cheap)

    Unfortunately blender still has a lot of complexities with certain types of animation that 3dsm and Maya can walk through, but projects like Big Buck Bunny aim to help address those issues and find ways around them
    its one of those FOSS areas that's still blossoming, in the same way architecture tools are still way more fiddly than things like Autodesk, but the more blossoming the better :)

    But your right, doing that with 3dsm would cost a bomb (the windows licences wouldn't kill it really, 6x Win2k8 standard aint exactly pricey)
     
    Certifications: MCSE, MCITP, VCP
    WIP: > 0
  8. Fergal1982

    Fergal1982 Petabyte Poster

    4,196
    172
    211
    Looks pretty good. Couple of questions though:

    can it be used for tasks other than graphic rendering?

    where can i place an order?
     
    Certifications: ITIL Foundation; MCTS: Visual Studio Team Foundation Server 2010, Administration
    WIP: None at present
  9. ffreeloader

    ffreeloader Terabyte Poster

    3,661
    106
    167
    The MS license allows a quad core under a single license? And can a standard 2008 server run in that type of environment, i.e. basically a cluster? Even if it can doesn't the MS license for their standard server pretty much exclude this type of application for the standard server package?

    However, figure even at 6 standard licenses just how much does that run now? Another $4800 or so? It more than doubles the cost of the entire project as it now runs.
     
    Certifications: MCSE, MCDBA, CCNA, A+
    WIP: LPIC 1
  10. ffreeloader

    ffreeloader Terabyte Poster

    3,661
    106
    167
    LOL. Yeah, you can order one. Order all the same parts from your favorite hardware vendor and build it yourself. :twisted:

    In my opinion the way that project is set up it's pretty much a graphics rendering project as the software (Dr Queue) that coordinates the processing is used only for controlling graphics rendering software such as Blender and Maya. That's what turns it into what is basically a parallel processing cluster.
     
    Certifications: MCSE, MCDBA, CCNA, A+
    WIP: LPIC 1
  11. Fergal1982

    Fergal1982 Petabyte Poster

    4,196
    172
    211
    but im too lazy.
     
    Certifications: ITIL Foundation; MCTS: Visual Studio Team Foundation Server 2010, Administration
    WIP: None at present
  12. Modey

    Modey Terabyte Poster

    2,397
    99
    154
    That is a cool project he's setup there. It really appeals to the geek in me. :)
     
    Certifications: A+, N+, MCP, MCDST, MCSA 2K3, MCTS, MOS, MTA, MCT, MCITP:EDST7, MCSA W7, Citrix CCA, ITIL Foundation
    WIP: Nada
  13. ffreeloader

    ffreeloader Terabyte Poster

    3,661
    106
    167
    LOL. Well, then going without is the price you're going to have pay then..... :biggrin

    I've actually started thinking about building one of these for work as we are very close to already overloading the servers that we have.

    It's really cool that he used a modified 6 drawer cabinet case designed for storage in a workshop to build this thing from. Go to the ikea website, click on the workspace link, and scroll down the page until you see the Helmer cabinet. :D
     
    Certifications: MCSE, MCDBA, CCNA, A+
    WIP: LPIC 1
  14. NightWalker

    NightWalker Gigabyte Poster

    1,172
    25
    92
    Do you run Seti on your PC?
     
    Certifications: A+, Network+, MCP, MCSA:M 2003, ITIL v3 Foundation
  15. BosonMichael
    Honorary Member Highly Decorated Member Award 500 Likes Award

    BosonMichael Yottabyte Poster

    19,183
    500
    414
    They allow any number of cores and processors under a single license. Each connecting device must have a CAL.

    Other MS server-based products are licensed per physical processor, regardless of the number of cores per physical processor: link
     
    Certifications: CISSP, MCSE+I, MCSE: Security, MCSE: Messaging, MCDST, MCDBA, MCTS, OCP, CCNP, CCDP, CCNA Security, CCNA Voice, CNE, SCSA, Security+, Linux+, Server+, Network+, A+
    WIP: Just about everything!
  16. Phoenix
    Honorary Member

    Phoenix 53656e696f7220 4d6f64

    5,749
    200
    246
    used too, and World community grid for a while
    decided to pack it in and let my cpu idle while i wasnt doing anything, give it a bit more life lol
    I've started to shut down non essential systems when not in use as well, making it moot
     
    Certifications: MCSE, MCITP, VCP
    WIP: > 0
  17. Phoenix
    Honorary Member

    Phoenix 53656e696f7220 4d6f64

    5,749
    200
    246
    Most windows licenses are per socket not per core, and standard supports dual sockets (so you could have 8 cores now, more in future)
    and as long as you kept the FOSS cluster services you wouldnt need the windows cluster package (not sure most windows render farms use MS clustering anyway, its not really designed for that)

    yes you would add about the cost of the project again (that's at retail costings) and you would still have a cheap as chips render farm if for whatever reason you required windows
    (wonder if a blender farm could render 3dsm/maya stuff, is there an open format for the raw data?)
     
    Certifications: MCSE, MCITP, VCP
    WIP: > 0
  18. nugget
    Honorary Member

    nugget Junior toady

    7,796
    71
    224
    That is just amazing.

    Thanks freddy. :thumbleft
     
    Certifications: A+ | Network+ | Security+ | MCP (270,271,272,290,620) | MCDST | MCTS:Vista
    WIP: MCSA, 70-622,680,685
  19. zebulebu

    zebulebu Terabyte Poster

    3,748
    330
    187
    Alpha.

    Nerd. :)

    Best line of the lot: "Who knew Ikea made their stuff ATX-compatible?' I LOLd at that one :biggrin
     
    Certifications: A few
    WIP: None - f*** 'em
  20. Phoenix
    Honorary Member

    Phoenix 53656e696f7220 4d6f64

    5,749
    200
    246
    Me too, and that damn things 39 bucks!
    39 bucks for a super computer case
    who would of thunk it?

    now if only someone did a highly efficient multi mobo psu :)
     
    Certifications: MCSE, MCITP, VCP
    WIP: > 0

Share This Page

Loading...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.