My research program seeks to understand the nature of the representations that underlie visual perception and memory. Specifically, I use principles of information theory and efficient coding to examine how encoding new information is constrained by prior knowledge and by perceptual processing. First, I will show evidence that working memory capacity is best quantified using principles of efficient coding rather than the number of independent items that can be remembered. I'll demonstrate that working memory representations depend on prior knowledge, and that items in working memory are not stored independently of one another. The effects of prior knowledge on encoding can be understood as a form of compression, and interactions between items suggest the display is represented hierarchically, including a summary of the display in addition to item-level information. Next, I will demonstrate that principles of efficient coding can reveal the representations of real-world objects in long-term memory. I will show evidence that observers are capable of storing thousands of detailed object representations, but only when prior knowledge allows them to treat the objects as conceptually distinct. Finally, I will describe ongoing work using formal models to discover the structure of visual representations. This approach broadly aims to understand the representations of real-world objects and scenes, allowing us to gain a deeper understanding of our integrated vision and memory system.