The Systems Operations Lead is responsible for all aspects of our technical infrastructure, including new build-outs, maintenance, and monitoring. They will take the reins of a high-traffic, multi-tiered environment consisting of over 70 production nodes. As the exchange backbone for the the secondary ticket market, some amazing stats include:
5.4 MM http requests per day, chick translates into 64 every second
12,000 DB transactions per second
9.5 MM DB rows returned per second
2 MM asynchronous jobs processed per day
Our API powers hundreds of websites and mobile applications, and over 700 ticket brokers rely on our system to buy, sell and manage their inventory, and by extension their business. This role requires someone who thrives in taking responsibility for all aspects of our operations to ensure high-performance and solid uptime, while also looking to the future on how we continue to scale horizontally and vertically.
Provide technical direction and guidance for how our company's infrastructure will be architected and managed.
Work directly with vendors such as our hosting company and monitoring staff to ensure that all our applications maintain high availability.
Taking ownership of our systems and knowing them inside and out, in order to troubleshoot infrastructure issues when they arise.
Keeping run books up-to-date and ensuring that normally-occuring issues can be mitigated or handled by on-duty operations support.
Ensuring that our systems are secure, redundant, and routinely backed up to provide failover if necessary.
Understanding what services we currently have running and knowing how to make them run better.
Keeping abreast of the latest technologies that can help us better serve our customers software needs.
3-5 Years of managing production systems
Solid understanding of Linux (Ubuntu preferred) and how to administer systems and their daemonized processes using upstart and init.d.
Strong knowledge in protocols such as HTTP, HTTPS and TCP.
Skilled in programming languages such as Ruby (preferred), Bash and SQL.
Experience in using Chef Server to configure and maintain servers.
Knowledge of Capistrano, and how to configure a deploy processes in a multi-tier environment.
Strong experience in PostgreSQL, including streaming replication, full-text search and data warehousing.
Experience in virtualization (VMware ESX along with vSphere preferred).
Skills in using source control such as Git
Knowledge in service oriented architecture.
Experience with Nginx, Passenger, HAProxy, Redis, Varnish, Graphite and ZooKeeper are a big plus.
Salary + Equity