There is a fundamental tension between storing discrete traces of individual experiences, which allows recall of particular moments in our past without interference, and extracting regularities across these experiences, which supports generalization and prediction in similar situations in the future. This tension is resolved in classic memory systems theories by separating these processes anatomically: the hippocampus rapidly encodes individual episodes, while the cortex slowly extracts regularities over days, months, and years. This framework fails, however, to account for the full range of human learning and memory behavior, including: (1) how we often learn regularities quite quickly—within a few minutes or hours, and (2) how these memories transform over time and as a result of sleep. I will present evidence from fMRI and patient studies suggesting that the hippocampus, in addition to its well-established role in episodic memory, is in fact also responsible for our ability to rapidly extract regularities. I will then use computational modeling of the hippocampus to demonstrate how these two competing learning processes can coexist in one brain structure. Finally, I will present empirical and simulation work showing how these initial hippocampal memories are replayed during offline periods to help stabilize and integrate them into cortical networks. Together, the work provides insight into how structured information in our environment is initially encoded and how it then transforms over time.