Sponsored Content
Hal Varian, the chief economist at Google, is famous for saying in 2010 that “the sexy job in the next 10 years will be statisticians.” Building on this idea, Thomas Davenport and D.J. Patil wrote “Data Scientist: The Sexiest Job of the 21st Century.”
Should you seek training in data science because the job is “sexy”? Probably not. You want to know that the job is in demand. Data scientists are in demand and so are data engineers.
Data scientists continue to be one of the fastest-growing occupations in the United States. According to the Bureau of Labor Statistics, the data scientist role boasts the 4th highest projected growth rate at 36% over the next decade (2023-2033), with an annual median pay of $108,020 in 2023.
There is also a high demand for data engineers. Indeed has included data engineers in their 2024 List of 10 Best Jobs in the US, with an average annual salary of $130,135.
The rise of generative AI and large language models has fundamentally transformed the technology landscape. Data scientist and data engineer roles are more critical than ever, particularly as organizations increasingly invest in these technologies. A 2024 Forrester survey reveals that 67% of AI decision-makers plan to increase generative AI investments in the coming year. This surge underscores the critical need for highly adaptable data scientists and data engineers with specialized skills in AI.
The Work of Data Science and Data Engineering
To remain competitive in today’s world, companies need to be informed by data and to use data in their day-to-day operations. That does not happen unless there are data engineers, technical professionals (software engineers, database administrators, cloud architects, and the like). These are people who can translate research results and data science models into systems that work.
The role of the data scientist is evolving with the emergence of capable generative AI and large language models. In addition to building new models from scratch, data scientists are integrating prebuilt models into the data pipeline. Many of these prebuilt models come from large technology firms. Building integrated, intelligent systems is the work of data engineering.
Data engineers have a key role to play at the beginning of every data science project. Data scientists depend on data engineers to gather and prepare data for analysis. Without data, there are no analyses, and no models to build and test.
Some analytics and modeling projects end with a written report to management or a display of results in a dashboard or presentation. Research findings guide management decisions.
Analysis and modeling projects need not end with a report. Many models are put into practice. They become the way a company conducts its business. Data engineers have key roles to play in building data science applications and implementing information systems.
Choosing the Right Job for You
Data scientists are like chameleons, changing their colors to match the business context. There are data scientists who could just as easily be called marketing researchers, financial analysts, or competitive intelligence professionals, depending on the work that needs to be done.
Analytics and modeling lie at the heart of data science, and many data scientists think of themselves as applied statisticians.
To serve the needs of today’s data-driven, data-intensive world, data scientists need to be multilingual, speaking the languages of information technology and business, as well as analytics and modeling.
Students interested in data science application development and systems implementation can specialize in data engineering and seek various information technology positions. Data engineers could just as easily be called software engineers, systems engineers, cloud architects, computer scientists, machine learning engineers, AI engineers, or development and operations (DevOpps) professionals. Job titles may vary from one company or one industry to the next, but the job opportunities are plentiful.
Reading job postings for data scientists and data engineers, you will find considerable overlap in the desired skillsets.
Data Scientist skills:
- Programming
- Cloud computing
- Database management
- Data visualization
- Probability and statistics
- Multivariate calculus and linear algebra
- Machine learning & deep learning
Data Engineer skills:
- Programming
- Extraction Transformation and Loading (ETL)
- System architecture
- Database design and configuration
- Interface and sensor configuration
Technology Management
Moving from the technology side to the management side of data science, presents additional opportunities. Organizations need people who can understand data science and data engineering, but also speak the language of business.
As with data science and data engineering, there are many jobs for technology managers. Someone needs to build and manage teams of data scientists and data engineers. Someone needs to make decisions about information infrastructure.
Communication skills are essential to technology management. There is a great need for people who can understand business problems as well as technical solutions to problems. There is a great need for people who can translate technical jargon into language that non-technologists can understand.
Whatever role is best for you—data scientist, data engineer, or technology manager—Northwestern University’s MS in Data Science program will help you to prepare for the jobs of today and the jobs of the future. The program is offered in a convenient and engaging online format. Students learn part-time at their pace, allowing for learning and growth without career interruption.