Data Intensive Scientific Thought

There is a common misconception in our society, especially among parents and students, that a computer science education is all about   programming . No, it is not.  Programming is certainly a part of computer science and computer scientists learn to program. In fact, they learn to program exceptionally well. But programming is not the   purpose   of a computer science education. It is simply the language computer scientists use to   express ideas, design systems, and solve complex problems . The legendary computer scientist Edsger W. Dijkstra famously noted: “ Computer science is no more about computers than astronomy is about telescopes. ” Just as an astronomer uses a telescope to study stars, galaxies, and the physics of the universe, a computer scientist uses programming to study, manipulate, and scale information. Programming is the canvas; computer science is   the architecture, creativity, and reasoning behind the painting . Reducing compute...

The narrative that AI will eliminate programming jobs has gained significant traction in both popular media and tech world in recent years. This narrative is based on a surface-level logic that suggests as a specific job gets easier and more efficient, you need fewer people to do it. I believe this narrative is fundamentally flawed; in reality, we are entering an era where the demand for specialized technical expertise in computing and software development will likely explode. In this blog post, I argue that we are witnessing a classic case of   Jevons Paradox  and AI will not decrease the demand for programmers, it will substantially increase it. Increased Efficiency Leads to Increased Demand: A Paradox Named after economist William Stanley Jevons , the paradox states that as the efficiency of a resource increases, the total consumption of that resource actually rises instead of falling. Why? Because increased efficiency makes the resource cheaper and more accessible, wh...

Large-scale AI models are considerable consumers of computing resources and energy, leading to a significant carbon footprint on our planet. Researchers estimate that training a single natural language processing model  can generate as much CO2e (carbon dioxide equivalent) as the annual emissions of 120 homes. AI workloads in data centers accounted for 15% of Google’s total electricity consumption -- 18.3 terawatt hours in 2021, which is comparable to the annual energy usage of the entire City of Atlanta. And this was well before the boom of generative AI technologies we have been witnessing over the last couple of years. Driven by the growing demands of large-scale data analytics and AI workloads, data centers  are projected to consume 3–13% of global electricity by 2030 -- a significant increase from just 1% in 2010. The computational demands of cutting-edge AI models are increasing 1,000-fold every three years, and AI could account for 14% of the world’s total carbon...

Figure 1: Estimation of expected total annual energy   consumption per IT industry in the period 2010–2030. The plethora of data generated by scientific applications, the Internet of Things, social media, and e-commerce fuel large-scale data analytics systems. As a result, data transfer over the Internet has been increasing each year exponentially and has already exceeded the zettabyte scale. With the increased data generation rate, the data movement’s carbon footprint is becoming an overwhelmingly critical problem, especially for data centers and wired access networks. It is estimated that information and communication technologies will use between 8% - 21% of the world’s electricity by 2030 . The estimation of expected total annual energy consumption per different IT industries in the period 2010–2030 is shown in Figure 1. The share of data centers and communication networks in the total IT power consumption is 69%. Among this share, the data transfers alone consume over a hun...

This semester, I'm offering the second instantiation of my seminar course on Green Computing and Sustainability .  The papers we plan to discuss this semester are listed below. This time, our focus will be Sustainable AI  and Sustainable Data Centers .  If you have recommendations on relevant papers to add to the list for our future seminars, please let me know.  Green and Sustainable Computing: Challenges and Opportunities  Assessing ICT Global Emissions Footprint: Trends to 2040  & R ecommendations The Sky is Not the Limit: Untapped Opportunities for Green Computing Sustainable Computing – Without the Hot Air The Odd One Out: Energy is not like Other Metrics  Sustainable AI Green AI Towards a Methodology and Framework for AI Sustainability Metrics Do Deep Learning Frameworks Have Different Costs? Adaptive and Energy-Efficient Architectures for Machine Learning: Challenges and Opportunities Energy-Efficient Deep Learning Inference on Edge Dev...

During my term as an  NSF  program director in the  Office of Advanced Cyberinfrastructure between 2020-2022, I had the opportunity to lead the development of  NSF’s Blueprint for a National Data and Software Cyberinfrastructure .  This blueprint document is publicly available to the community and provides a forward-looking vision for a robust, secure, trusted, performant, scalable, and sustainable data and software cyberinfrastructure (Data and Software CI) ecosystem to enable and accelerate science and engineering research. This blueprint was prepared based on a comprehensive analysis of existing NSF programs and a wide range of input from the community via advisory bodies, requests for information (such as Data-Focused   Cyberinfrastructure Needed to Support Future Data-Intensive Science and Engineering Research  and Future Needs for Advanced Cyberinfrastructure to Support Science and Engineering  Research ), community...

Software applications are considerable consumers of computing resources, leading to a significant carbon footprint. Recent estimates suggest that Information Technology accounts for approximately 11% of global energy consumption . Projections indicate that by 2030, data centers alone could account for 3–13% of global electricity use, a stark increase from the 1% recorded in 2010, propelled by escalating demands and emerging trends like large-scale AI workloads . The software industry is responsible for about 3% of global carbon emissions , which is very close to that associated with the aviation industry , and it could deteriorate if left unaddressed. Researchers have estimated that training a single machine learning algorithm for natural language processing may emit as much CO2e (carbon dioxide equivalent) as a small town with 120 homes would emit in a year . Another study reported that AI workloads in data centers made up 15% of Google’s total electricity consumption, which was 18...