Burning Man, the World Series, and New York Comic Con – what do these events have in common? They’ve all recently failed at selling tickets online.
More recently Sundance Film Festival issued a public apology after their ticketing system crashed under enormous strain due to demand.
Ticketing fails don’t just affect events of mass scale either. Every day, event ticketing platforms succumb to unwavering demand, eventually bringing registration to a standstill. Picatic was no exception.
This past summer we were approached by a local arts association in Vancouver. They organize events around the Burning Man community and its loyal following here in British Columbia, Canada. I sat down with two members of their organizing committee to learn more about their unique needs. We spoke at length about ticketing and the upcoming challenge we’d face with the kind of demand they were expecting.
We have worked with events in the past with large bursts of traffic and thought we knew how our systems scaled. So we said, “No problem”.
Our team was confident that spinning up additional servers on the day of the ticket sale would handle the flood of traffic to the registration page.
That day arrived, and over 8,000 unique clients (IP/device) attempted view the event page in the span of 45 minutes. To compound this our requests per second went from a daily peak of 9/second to over 40/second, to at times 100/second …
But we also made a promise. We promised that by the time this event organizer had their next round of ticket sales, we’d have addressed the issues on-hand and be ready to service every single person who comes to the event page to register.
And that’s exactly what we did.
In order to handle the type of demand we saw with this event, we knew we’d have to reduce the number of registrants in ticket checkout at any given time. This is the most technically complicated process within Picatic and it was the main cause for the bottleneck and eventual crash of our system.
To do this we introduced a queueing system, known internally as Project Q. This solution would allow any number of potential registrants to access the event page and select the number of tickets they’d like to order. Once their order is submitted, they would then be placed into a queue. From here a set number of registrants are moved into checkout. As they complete their orders, new registrants from the queue are placed into checkout. Yes, we created a virtual line.
This allows us to control the exact number of registrants within checkout at any time. As you all know, queueing isn’t new to online ticketing. In fact, experts in queueing theory (yes, they exist) say it got its start about 100 years ago necessitated by a new technology: the telephone. So, chances are you’ve waited in a queue at one point or another. You’ve also probably had some good and bad experiences waiting in a queue. When building out Project Q for Picatic events, we knew three things about queueing psychology:
- We hate it when we expect a short wait and then get a long one
- We really hate it when someone shows up after us but gets served before us
- We get bored when we wait in line
In our first iteration of Project Q we decided to tackle the first two points on this list. To do this we understood that communication on the queueing page was key. We needed to answer three questions that we assumed registrants in the queue would have:
- What am I looking at right now?
- What happens next?
- How long will I be in line?
Studies show that we’re much more patient when we’re given an idea of how long we’ll be waiting. To accomplish this, we show each registrant their spot in the queue as well as how many purchasers are both ahead and behind them. We answer the questions “what am I looking at right now?” and “What happens next?” with a rollover “What is this?” call to action.
As we had promised our client, our queueing system was built and ready to go for their next ticket sale. We thoroughly tested our new tool by recreating the local environment that had caused the crash about a month earlier. All tests were successful and we were eager to make things right for our client.
On the day of the ticket sale, we watched as registrants entered the queue and proceeded successfully through to checkout. Our goals were simple:
- Keep the event page from crashing
- Move purchasers from ticket selection, to the queue, to checkout
- Avoid queueing rage
Throughout the ticket sale, the event page remained up and over 500 tickets were purchased in under 30 minutes.
There were slight issues along the way and potential registrants needed to be manually moved from the queue into checkout. We also uncovered an issue that was allowing more than one registrant to be on the same timestamp and thus in the same spot in queue.
Since this particular event, our queueing system has been used a handful of times and we continue to iterate and fine-tune the feature.
The hurdles of selling tickets to in-demand events can be massive. We’re not professing to have built the perfect tool to sell tickets to prolific events like Burning Man or The World Series. However, we do believe we’ve built the beginnings of something scalable that will eventually bring Picatic to that level.