Serverless is clearly on the radar of data engineers and architects. In a recent survey, we found 85% of respondents already had parts of their data infrastructure in one of the public clouds, and 38% were already using at least one of the serverless offerings we listed. As more serverless offerings get rolled out—e.g., things like PyWren that target scientists—I expect these numbers to rise.
We had a great conversation spanning many topics, including:
A short history of cloud computing.
The fundamental differences between serverless and conventional cloud computing.
The reasons serverless—specifically AWS Lambda—took off so quickly.
What can data scientists and data engineers do with the current generation serverless offerings.
What is missing from serverless today and what should users expect in the near future.