A bookstore from a port in the wall

The situation

Back then, almost nobody had heard of Amazon. The requirements I was handed were short: here is an RS-232 port, and here is the data you can reach through it. Figure out how to get at it, and use it to build a bookstore. If a book exists in that data, sell it, and don’t worry about quantities. The store would carry any publication listed in the ISBN database, not just the titles someone had found time to enter.

The work came to me through a consulting firm. I was one of two architects and developers selected for the project, and each of us was asked to design and build a proof of concept.

Why it was harder than it sounds

Each stage between the source and the storefront had its own requirement.

• The source sat behind a serial line. The ISBN records were reached over an RS-232 serial connection, so the first job was building a dependable pipeline to pull them across.
• The records were not ready to use. What arrived had to be parsed and normalized before it was consistent enough to search, sort, and sell from.
• A web store needs a database, not a feed. The data had to be structured relationally so a website could query it on demand.
• The web had almost none of today’s tools. There was no XMLHttpRequest and no ASP.NET. A shopping cart that saved to a database was something you built from scratch.
• A catalog that depends on typing does not scale. The point was a live, dynamic catalog. Any step that needed a person to key in records would defeat it.

The approach and the decisions behind it

Go to the source directly.

I established an RS-232 serial communication pipeline to retrieve ISBN records straight from the database, so the catalog was built from the ISBN records themselves.

Clean the data on the way in.

Every record was parsed and normalized for consistency before it was stored, so the website worked from one clean, uniform set of records.

Store it where the web can ask questions.

The normalized records were structured into a relational SQL database built for web-based querying: the shape a browsable, searchable storefront needs.

Let the data decide what’s for sale.

If a book was in the ISBN data, the store could offer it. Nobody had to choose titles or count stock, so the catalog stayed as current as the data itself.

Give it a storefront.

On top of the database I built the visual side: a catalog to browse and a shopping cart that saved each order to the database.

Write it down for the next stage.

I documented the design and operational details so the system could be carried forward and scaled beyond the proof of concept.

How it works

The proof of concept was a pipeline with one job: turn a stream of book records into a catalog a website could sell from.

1. Retrieve. ISBN records come across the RS-232 serial connection from the database.
2. Parse and normalize. Each record is broken into its parts and made consistent with every other record.
3. Store. The cleaned records are written into a relational SQL database designed for web queries.
4. Serve. The online store browses and sells from that database, and any book in the data can go into the shopping cart.

Outcome

In the end the choice was not one design or the other: both were selected, and the other architect’s work was pulled in alongside mine. I never learned how his was built, so I can only speak to my half. By today’s standards it was crude. At the time, almost nobody was doing it.

My role

I was one of the two architects and developers on the project, engaged through a consulting firm rather than hired by Amazon directly. I designed and developed my proof of concept, from the port in the wall to a working storefront: the serial pipeline to the ISBN database, the parsing and normalization, the relational SQL structure, the catalog and the shopping cart. I documented the design and its operation so it could be carried forward. It was a very small part of what Amazon became, and I’m glad to have had it.