We’re looking to hire our first engineer focused on ops automation. This is primarily a software engineering role, although to be successful you’ll need to have roots in and a passion for systems administration. At blist systems engineers develop software that automates as much of operations as possible. The goals are fully lights out operation via automated systems deployment, imaging, monitoring, error detection and healing.
blist is a well capitalized startup developing database as a service, operated at Internet scale. We’re solving some really interesting challenges and have a terrific team of passionate engineers. If you are interested in joining us, send your resume along with the solution to the following challenge:
Assume you have a network comprised of 1,000 servers in 10 different data centers - 100 servers in each data center. The data centers are in multiple time zones. Write a centralized script that runs on the Linux PC in your office, which identifies the server in the network which was most recently rebooted. The output should identify the server, the data center it’s in, the date & time when it was last rebooted and how long the script ran in order to find the results. Your script must finish in less than 5 minutes (300 seconds). At any time 2% or 3% of the servers will be offline.
Your solution should include:
* A description of your assumptions about network topology - how your Linux PC connects to each server
* Any other base assumptions you make about the servers in the network
* A description and, if appropriate, the layout of any configuration file(s) you’ll need to solve the problem
* Commercial quality perl or python code that solves the problem and prints the results
Ability to design and develop production quality automation code, preferably in python but at minimum in perl
Solid understanding and experience with PXE boot based automated image deployment
Good familiarity with Xen or other virtualization software
Good familiarity with systems monitoring software (Nagios, etc.)
Broad Unix/Linux systems administration experience
Some experience in database administration is helpful, but not mandatory
Passion for distributed computing