"COLLECTIVE INTELLIGENT BRICKSmany thermal ideas. We thank Larry Mok, Mark Ritter, and Al Widmer from the IBM Thomas J.Watson Research Center and Professor Martin Graham of the University of California atBerkeley for many enlightening discussions. We also thank our intern, Ekpe Okorafor, whoworked on the base software. 1 Between 2001 and 2005. these data center issues and the ideas described here were discussedduring some 70 customer visits to the IBM Almaden Research Center. 2 Bricks are not required to have the physical shape commonly associated with the word. Forexample, a system using blades or 1U servers qualifies as brick-based if the other criteria arefulfilled. (See the next section.) 3 Bricks should not be confused with thin clients managed by a central server. 4 There is no relation to the IBM Cell Broadband Engine** processor. 5 For a storage server, system density may be measured in units of storage capacity/floor space. 6 It should be emphasized that high serial capacitance is desirable in a capacitive coupler. This isthe opposite of the common situation in high-frequency electronics. 7 The prototype has a single management processor, which is a single point of failure. A largersystem would employ N management processors with a fail-over scheme. * Trademark, service mark, or registered trademark of International Business MachinesCorporation. ** Trademark, service mark, or registered trademark of Sony, Egenera, Inc., Advanced MicroDevices, Inc., InfiniBand Trade Association, Linus Torvalds, Cluster File Systems, Inc., orSPARC International, Inc., in the United States, other countries, or both. DEPT OF IT, PDCE 2009-2010 Page 43 COLLECTIVE INTELLIGENT BRICKS10. REFERENCES 1. N. Allen, "Don't Waste Your Storage Dollars: What You Need to Know," Research Note,Gartner Group Inc., Stamford, CT 06904, March 2001. 2. L. Wood, "The Hidden Costs of Unmanaged Storage," Enterprise Storage Forum.Com, July 7,2003; see http:// www.enterprisestorageforum.com/technology/features/article.php/11176_2231701_2. 3. C. Fleiner, R. B. Garner, J. L. Hafner, KK Rao, D. R. Kenchammana-Hosekote, W. W.Wilcke, and J. S. Glider, "Reliability of Modular Mesh-Connected Intelligent Storage BrickSystems," IBM J. Res. & Dev. 50, No. 2/3, 199-208 (2006, this issue). 4. K. Fernandez, C. Fleiner, R. Garner, H. Huels, M. Ries, and W. Wilcke, "System and Methodfor Providing Cooling in a Three- Dimensional Infrastructure for Massively ScalableComputers," U.S. Patent Application 20050152114, filed July 14, 2005. 5. R. Schmidt, "Liquid Cooling is Back," Electron. Cooling 11, No. 3, 34-38 (August 2005). 6. Datacom Equipment Power Trends and Cooling Applications, M. Geshwiler, Ed., AmericanSociety of Heating, Refrigerating and Air- Conditioning Engineers, Inc., Atlanta, GA, 2005;ISBN 1-931862-65- 6. 7. P. Sarkar, K. Voruganti, K. Meth, O. Biran, and J. Satran, "Internet Protocol Storage AreaNetworks," IBM Syst. J. 42, No. 2, 218-231 (April 2003). 8. R. B. Garner, W. Wilcke, B. Rubin, and H. Kahn, "A Scalable Computer System HavingSurface-Mounted Capacitive Couplers for Intercommunication," U.S. Patent Application20040066249, filed October 2002. 9. W. Wilcke, R. Williams, R. Garner, and C. Fleiner, "Mechanism for Self-Alignment ofCommunication Elements in a Modular Electronic System," U.S. Patent Application 10-987,901,filed November 12, 2004. DEPT OF IT, PDCE 2009-2010 Page 44 COLLECTIVE INTELLIGENT BRICKS10. J. D'Ambrosia, S. Rogers, and J. Quilici, "XAUI-An Overview," white paper; seehttp://www.techonline.com/community/ related_content 121173. 11. A. X. Widmer and P. A. Franaszek, "A DC-Balanced, Partitioned- Block, 8B/10BTransmission Code," IBM J. Res. & Dev. 27. No. 5. 440^t51 (1983); see alsohttp://www.interfacehus.com/ Definitions.html. 12. S. Glasstone, Principles of Nuclear Reactor Engineering. D. Van Nostrand Company, NewYork, 1955. 13. T. Sterling, P. Messina, and P. H. Smith, Emitting Technologies for Petaflops Computi\\ng,The MIT Press, Cambridge, MA, 1995; ISBN 0-262-69176-0. 14. K. Rao, J. Hafner, and R. Golding, "Reliability for Networked Storage Nodes," ResearchReport RJ-10358, IBM Almaden Research Center, San Jose, CA 95120, September 2005. 15. D. A. Patterson, G. Gibson, and R. H. Katz, "A Case for Redundant Arrays of InexpensiveDisks (RAID)," Proceedings of the ACM SIGMOD International Conference on Management ofData, June 1988, pp. 109-116. 16. J. L. Hafner, "WEAVER Codes: Highly Fault Tolerant Erasure Codes for Storage Systems,"Proceedings of the 4th USENIX Conference on File and Storage Technologies, December 15,2005. 17. IBM Journal of Research and Development, Vol. 49, No. 2/3, 2005; special issue on BlueGene; see also http:// www.research.ibm.com/bluegene/. 18. IBM Corporation,"General Parallel File System," 2005; see http://www- 03.ibm.com/servers/eserver/clusters/software/gp.fs.pdf. 19. M. L. Massie, B. N. Chun, and D. E. Culler, "The Ganglia Distributed Monitoring System:Design, Implementation, and Experience," Parallel Computing 30, No. 7, 817-840 (July 2004);see also http://ganglia.sourceforge.netl. DEPT OF IT, PDCE 2009-2010 Page 45 COLLECTIVE INTELLIGENT BRICKS20. C. G. Bell, J. C. Mudge, and J. E. McNamara. Computer Engineering; A DEC View ofHardware Systems Design, Butterworth- Heinemann, Newton, MA. 1978. 21. G. Bell and J. Gray, "What's Next in High-Performance Computing?," Commun. ACM45,No. 2, 91-95 (February 2002). 22. C. Chao, J. Wilkes, D. Jacobson, B. Sears, R. M. English, and A. A. Stepanov, "DataMeshArchitecture 1.0," Technical Report HPL- 92-153, Hewlett-Packard Computer SystemsLaboratory, PaIo Alto, CA 94304, December 1992. 23. Y. Saito, S. Frlund, A. Veitch, A. Merchant, and S. Spence, "FAB: Building DistributedEnterprise Disk Arrays from Commodity Components," Proceedings of the 1 lth InternationalConference on Architectural Support for Programming Languages and Operating Svstems(ASPLOS), 2004, pp. 48-58. 24. G. R. Ganger, J. D. Strunk, and A. J. Klosterman, "Self-* Storage: Brick-Based Storage withAutomated Administration," Technical Report CMU-CS-03-178, Carnegie Mellon University,Pittsburgh, PA 15213, August 2003. 25. E. K. Lee and C. A. Thekkath, "Petal: Distributed Virtual Disks," Proceedings of the 7thInternational Conference for Architectural Support for Programming Languages and OperatingSystems (ASPLOS), 1996, pp. 84-92. 26. J. Kubiatowicz, D. Bindel. Y. Chen, S. Czerwinski, P. Eaton, D. Geels, R. Gummadi, S.Rhea, H. Weatherspoon, W. Weimer, C. Wells, and B. Zhao, "OceanStore: An Architecture forGlobal-Scale Persistent Storage," Proceedings of the 9th International Conference onArchitectural Support for Programming Languages and Operating Systems (ASPLOS), 2000, pp.190-201. 27. D. Nagle, D. Serenyi, and A. Matthews, "The Panasas ActiveScale Storage Cluster- Delivering Scalable High Bandwidth Storage," Proceedings of the ACM/IEEE SupercomputingConference, 2004; see http://www.sc-conference.org/sc2004/schedule/pdfs/ pap207.pdf. Received July 5, 2005; accepted for publication August 18, 2005 DEPT OF IT, PDCE 2009-2010 Page 46 COLLECTIVE INTELLIGENT BRICKSWinfried W. Wilcke IBM Research Division, Almaden Research Center, 650 Harry Road, SanJose. California 95120 ([email protected]). Dr. Wilcke is a program director at theIBM Almaden Research Center, where he started the Intelligent Bricks project and IceCubeimplementation in 2001. He was a Senior Manager at the IBM Thomas J. Watson ResearchCenter, responsible for Victor/ Vulcan research, later commercialized in IBM SPsupercomputers. As Director of Architecture (later Chief Technical Officer) of HaL computers,he was deeply involved in the creation of the 64-bit SPARC** architecture. Dr. Wilcke receiveda Ph.D. degree in nuclear physics and previously worked at the University of Rochester and atthe Los Alamos and Lawrence Berkeley National Laboratories. Robert B. Garner IBM Research Division. Almaden Research Center. 650 Harry Road. San Jose,California 95120 (robgamdii-us.ihin.com). Mr. Garner received his M.S.E.E. degree fromStanford University. Before joining the IBM Research Division in 2001 to work in the IceCubeproject, he worked at Xerox Palo Alto Research Center. Sun Microsystems, and BrocadeCommunications. Claudio Fleiner IBM Research Division, Almaden Research Center, 650 Harry Road, San Jose,California 95120 ([email protected]). Dr. Fleiner received a Ph.D. degree in computer sciencefrom the University of Fribourg, Switzerland. Prior to joining the IBM Almaden ResearchCenter, he worked at the IBM Zurich Research Laboratory and Transmeta. Dr. Fleiner's researchinterests include large storage systems, distributed computing, and computer networks. Dr. Fleiner received a Ph.D. degree in computer science from the University of Fribourg,Switzerland.He was a Senior Manager at the IBM Thomas J. Watson Research Center, responsible for Victor/Vulcan research, later commercialized in IBM SP supercomputers. As Director of Architecture(later Chief Technical Officer) of HaL computers, he was deeply involved in the creation of the64-bit SPARC** architecture. Dr. Wilcke received a Ph.D. degree in nuclear physics andpreviously worked at the University of Rochester and at the Los Alamos and Lawrence BerkeleyNational Laboratories.DEPT OF IT, PDCE 2009-2010 Page 47 COLLECTIVE INTELLIGENT BRICKSRichard F. Freitas IBM Research Division, Almaden Research Center, 650 Harry Road, SanJose, California 95120 ([email protected]). Dr. Freitas received a Ph.D. degree inelectrical engineering and computer science from the University of California at Berkeley. Dr.Freitas is currently exploring the use of nonvolatile solid-state memory technology for storagesystems. Richard A. Golding IBM Research Division, Almaden Research Center. 650 Harry Road. SanJose. California 95120 ([email protected]). Dr. Golding received a Ph.D. degree for work onweak-consistency distributed systems. He was previously an architect at Panasas and aresearcher at Hewlett-Packard Laboratories. He currently leads the Collective Intelligent Brickssoftware effort. Joseph S. Glider IBM Research Division, Almaden Research Center, 650 Harry Road, San Jose,California 95120 ([email protected]). Mr. Glider is a Senior Technical Staff Memberand manager in charge of the Intelligent Bricks Storage Software project at the IBM AlmadenResearch Center. He received a B.S.E.E. degreSource: IBM Journal of Research and DevelopmentRelated Articles? Set-Top Box Manufacturers Select Spansion MirrorBit(R) SPI Multi-I/O Flash Memoryto Simplify Designs and Lower Overall System Costs? C.E. Thermal Systems Introduces the Energy-Efficient Heat by Design Picture HeaterProduct Line? Magnetek, Inc. Sells Telecom Power Systems Assets and Business to Myers PowerProducts, Inc.? Bosch Introduces Solar Thermal Water Heating Solutions to U.S. Market? Microsoft Announces Public Beta Release of System Center Data Protection ManagerVersion 2DEPT OF IT, PDCE 2009-2010 Page 48 "