Definition
The ability to analyze and improve the efficiency of code so that it runs with optimal complexity, creating systems that store and transmit data efficiently.
The ability to analyze and improve the efficiency of code so that it runs with optimal complexity, creating systems that store and transmit data efficiently.
Efficiency and performance are critical for our software. Our back-end systems need to organize data efficiently and enable efficient queries. Different types of services access our back-end systems by running algorithms that extract insights from large amounts of data. Our front-end systems provide different types of visualizations in real time. It's imperative that both code and data are as efficient as they can be to create a responsive experience for users.
Palantir engineers need to determine which components in a large system are critical for efficiency, and where can we make trade-offs. When should we replicate data in multiple storage formats that each optimize for a different type of query versus optimizing for space? How can we take advantage of servers with very high memory by keeping a part of the data in memory for fast access: what data do we keep, how will it be synchronized, and how can it be accessed most efficiently? How can we detect and optimize bottlenecks: when is processor time optimization critical, and when is performance I/O bound?
Master the fundamentals, but remember real-world constraints. Some advice: