Designed To Query Everything - Teradata's Massively Parallel Revolution

In 2013 I read The Soul of the New Machine by Tracy Kidder that tells the story of Data General's first 32-bit machine. Normally, I'm rarely enthusiastic about reading books that focus on computing technology but this was different. Kidder's telling of one slice of the Data General story shone a light as much on the personalities, industry challenges and organisational inertia that played a part in the creation of 'the machine' than the technological challenges the thirty or so Microkids and Hardy Boys faced under the leadership of Tom West at the end of the 1970's.

The story of how a small group of tenacious, innovative engineers brought about the creation of Data General's Eagle 'against all the odds' is a common theme in the early years of computing and as I turned each page and absorbed the devotional obsession of Tom West and his engineering team, my thoughts kept straying to the genesis of another computer that also began to take shape towards the end of the 1970's - the Teradata Database Computer. 

The beginnings of Teradata is, in many ways, an even more familiar creation story: a small group of gifted individuals with a vision, the ubiquitous start-up in a garage, a supportive venture capital partner that provided guidance as well as capital, and the determination and belief of its founders to bring a new way of computing to the market. But the creation of the Teradata Database Computer is not just a story about a machine that challenged the status quo but also a story about the people that gave it life and continued to develop it ever since.

But in many ways it's a relatively unique story, as very few fledgling companies had to manage and resolve so many breakthrough moments and industry firsts in order to design, build, sell and ship the first system. It was one of the first to use the Intel 8086 microprocessor and the Winchester hard drive at scale, paving the way for affordable and powerful systems based on commodity components. Teradata created an inter-process communications fabric that is recognisable in today's generation of machines whilst pioneering the use of relational parallel database technology when the industry was highly sceptical about the performance and relevance of the relational model. From its inception, Teradata's database computer heralded the start of the integrated hardware and software trend that began to break the need for ever increasing, monolithic central mainframe systems.  

Having hurdled complex technical challenges, Teradata went further than any of its peers to ensure its technology delivered. Teradata was one of the first companies to seriously crowd-source elements of its development. Before the first DBC/1012 was shipped to a customer in 1983, prospective customers like Merrill Lynch, Standard Oil and Proctor & Gamble lent their own people to Teradata to road test the ‘database computer’ and provide design input. The Partners programme meant that Teradata development could focus on real world challenges to help prospects understand the capabilities and the potential value of Teradata.

The Teradata DBC/1012 was the first 'appliance', a form factor that is so widespread today, and provided an affordable, scalable, parallel processing system that was designed to transform the way organisations could use data to make faster and more accurate business decisions. The database machine arrived just at the right time to service the demand for data analysis that new and maturing business methodologies like Michael Porter’s 4 P’s and Peter Senge’s Fifth Discipline were calling for. When that first system was shipped in 1984, it was a moment that catapulted the science of decision support from the back-room world of operations research into everyday business processes, spawning the $128bn data and analytics industry that we see today. Because of Teradata, the explosion of high volume data analysis fuelled the birth of competencies such as database marketing, category management and risk modelling that changed the corporate landscape. 

Businesses that were early adopters of such data driven practices began sweeping aside many long established brands. The data genie was out of the bottle and today’s increasingly sophisticated data-intensive analytical methods had their genesis in the wake of Teradata’s affordable, scalable, massively parallel revolution.

In 2019 this incredible journey will mark over 40 years of continuous evolution and leadership, building on the industry firsts and zest for innovation that have been in the DNA of Teradata since its inception. Unlike many tech companies that came into being in the 1970's, Teradata is as relevant today in the world of Cloud, AI and the Internet of Things as it was in the 1980's when demand for management information and detailed analysis was emerging.

As we approach Teradata's 40th anniversary, it is the right time to tell the story of Teradata’s legacy and its relevance to the continued development of data and analytics. It’s important for those, like myself, that have been on the journey and know how exciting the real story is. But it is also important for the industry to open a window on an important moment at the birth of data warehousing and business intelligence rather than continue to consume the myths and legends it has created to sell today’s dreams. The contribution of Teradata's alumni to the field of data science needs to be writ boldly alongside the likes of Tukey, Codd, Shannon, Cray, Treybig, Gray and Piatetsky-Shapiro.

With insight from the founders of Teradata, the employees that were there as the company matured, those of us that have followed since and the customers that bought into a brand new technology, the story of Teradata's creation and evolution needs to be told to faithfully document the achievements of the company and the Teradactyls that made it possible.