Spring
Follow
Spring AI and TypeSafe Jev: Fast, Cheap, Structured Decisions
Spring AI TypeSafe integrates TypeSafe AI's Jev API to provide fast, cheap, and structured decision-making components for AI applications. It bypasses traditional prompt templating and JSON schema parsing, instead offering a client that takes a state and typed questions to return structured answers. The system handles three primitive question types: Noul for yes/no, Choice for selecting an option, and Score for ranking on a rubric. These primitives return precise values, probabilities, and confidences, enabling granular evaluation.The service is designed for speed and cost-effectiveness, significantly outperforming typical chat model calls in benchmarks for both performance and price. This allows for checks at various stages of AI pipelines. The core idea is to compose atomic questions in code, allowing the service to process them in parallel against a single state observation.This capability is exemplified in the LLM-as-a-Judge scenario, where JevJudge can be built with specific criteria and thresholds to evaluate model outputs. Confidence scores provide an additional axis of information, indicating the certainty of an answer based on its distribution. The JevSelfRefineAdvisor leverages this judgment system to implement self-correction loops, retrying model calls with feedback when criteria are not met.Furthermore, JevGuardrailAdvisor acts as a safety layer, screening both user inputs and model outputs for predefined hazards. It does not retry but rather blocks or flags based on severity. The system's output is structured, offering classifications, scores, and decisions without generating prose.The TypeSafe AI primitives are Noul (yes/no, returning a truth value between 0 and 1), Choice (selecting one option, returning the chosen label and probabilities for all options along with confidence), and Score (placing on an ordered rubric, returning a continuous value and associated probabilities). Option descriptions and clear instructions for these primitives significantly improve performance and confidence. The project is available as a Maven Central starter and can be used with or without Spring Boot.