Agents struggle to learn from past experiences in long-running real-world tasks. Existing memory methods either record exhaustive actions or only successful workflows, failing to distill higher-level reasoning and neglecting failures. ReasoningBank addresses this by distilling useful insights from both successful and failed experiences for agent self-evolution. It creates structured memories with titles, descriptions, and distilled reasoning steps, decision rationales, or operational insights. The memory workflow involves continuous retrieval, extraction, and consolidation, with an LLM-as-a-judge assessing trajectories. Unlike other methods, ReasoningBank actively analyzes failures to learn preventative lessons and strategic guardrails. It integrates with memory-aware test-time scaling (MaTTS), using parallel and sequential scaling to generate richer learning signals. MaTTS allows agents to explore extensively, distilling high-quality memories through self-contrast and iterative refinement. Evaluation on web browsing and software engineering benchmarks shows ReasoningBank improves both agent effectiveness (higher success rates) and efficiency (fewer task steps). With MaTTS, performance is further boosted, demonstrating a strong synergy between memory and scaling. The system also exhibits emergent strategic maturity, evolving simple rules into complex, preventative logic structures over time. ReasoningBank offers a powerful framework for continuous learning in LLM-based agents, highlighting memory-driven experience scaling as a crucial frontier.