Skip to content

Theseus

Theseus is a set of software libraries created by WGBH to help make building and maintaining websites easy.

At its core, Theseus is a set of schemas that represent common types of data we deal with on our sites: pages, images, videos, slideshows, rich text, menus, cards, social media accounts, events, places, people, organizations, shows, seasons, episodes, sponsorship, and so on. These schemas are defined using Pydantic. Making a project using Theseus only requires using these shared schemas. Pydantic allows us to easily export our schemas in JSON-Schema format, which in turn enables us to generate code in a variety of languages. Specifically, we generate Typescript interfaces for use in the wgbhx project.

We use FastAPI along with the Pydantic schemas to create APIs. FastAPI will automatically generate OpenAPI (f.k.a Swagger) documentation, which describes API endpoints, their available HTTP methods, and input/output definitions in the JSON-Schema format via integration with Pydantic.

Most of the websites we build consist of content created by the national productions (Antiques Roadshow, Nova, Masterpiece, American Experience, and Frontline). With the exception of Nova Labs and Roadshow's ticketing lottery, the content on these sites is created by the production staff. We use Wagtail as a CMS, and the theseus-wagtail project contains models and StreamField blocks that are designed to make representing CMS content as Theseus schemas seamless.

Why 'Theseus'?

Theseus is a figure in Greek mythology. You may remember that he was responsible for killing the Minotaur. The project's name is actually inspired by his ship - or more specifically, the thought experiment that considers the ongoing maintenance of his ship.

Background

Our websites have a long lifespan. Most of them have been around since the early days of the web, going back to the early-mid 1990s. Much of the content that was created for our sites decades ago is still around in some form. Unfortunately, each production's website has always been created (and recreated) as a unique project, despite the obvious similarities between them. Something as basic as an image has always been handled in a way unique to a given iteration of each site1.

The Theseus project was created to help steer us towards a more maintainable future. The original idea was: as we got the opportunity (i.e. funding/capacity) to work on our websites, we'd replace parts of them to function like the best version of that 'part' from another site, just as the Ship of Theseus was rebuilt gradually over time.

Another problem we were discussing around the same time was a lack of communication between back end and front end teams. Historically, the back end developer(s) for a site would (understandably) design data structures by considering how to structure a relational database. The data would then be gathered together as context objects (Python dictionaries) for use in a Django template or serialized into JSON for use in an API. The design of these data structures was purely a back end exercise, and we'd sort of throw the data at the front end team and let them sort it out. At the same time, the design team would create mockups for page layouts, with little to no communication with developers about the data being presented. Front end developers would then try to force sometimes vague and unpredicable data structures from the back end team onto often vague and sometimes nonsensical assets from the design team.

Pydantic was gaining popularity around the same time, which led to discussions around schema-driven development. It seemed obvious that establishing data structures communally in a way that can be understood by all parties could help start chipping away at the mountain of technical debt and hordes of bugs our usual workflow had created.2

Most of the schemas in Theseus were written by our internal developer team - a group of front end, back end, and full stack developers who were all familiar with our websites and the types of data we deal with regularly. We met weekly for about a year to discuss what was worth modeling, and then writing and rewriting schemas.

  1. Actually, there was a project about 10 years ago called Bedrock that sought to solve this issue, but the methodology and workflow was flawed and only two Bedrock sites remain today, and they are unmaintained. ↩

  2. This article was also a major source of inspiration. ↩