Hybrid intelligence makes the ... Note

Hybrid intelligence makes the device part of enterprise AI architecture

Endpoint and AI strategies are merging as AI workloads become more distributed across capable devices. IT leaders must now consider how device decisions impact future AI platforms. Enterprise AI has focused on platforms, models, and cloud services, but AI PCs introduce endpoints as crucial execution environments. Different AI workloads necessitate varied execution environments, with sensitive or latency-critical tasks benefiting from local processing. Cloud models remain ideal for large-scale or complex reasoning tasks. Hybrid intelligence integrates local and cloud resources, making the device an active part of the AI platform. Agents, operating over time and accessing data, further complicate architectural decisions. This intersection impacts application design, security, and cost, requiring collaboration between endpoint, security, and application teams. Organizations can standardize devices while offering specialized configurations for demanding AI workloads. Existing cloud-based AI solutions can be extended by local inference, distributing processing between device and cloud. Failure and fallback strategies are crucial for both endpoint and cloud-based workloads, impacting security, privacy, and reliability. Agentic applications require controls for file access, tool usage, and activity across environments. Technologies like Microsoft Execution Containers and Microsoft Agent 365 aim to manage agent activity as part of an enterprise. Workload placement depends on factors like response time, privacy, connectivity, and available resources. Hybrid runtimes can dynamically route tasks to appropriate local or cloud resources based on real-time conditions. Software development showcases this by leveraging cloud models for complex problems and local models for targeted tasks. Local execution requires compatible models, runtimes, application paths, and sufficient system capacity. Windows ML provides a foundation for running supported models across various hardware resources. Incorporating local inference means hardware becomes an integral part of the architecture, influencing performance and cost. Device planning should consider employee roles and potential dedicated local infrastructure. Field inspection workflows and copilot illustrate hybrid intelligence by combining local and cloud capabilities. Local AI workloads must be evaluated within the context of overall device activity. Device manufacturers are designing hardware to support evolving AI landscapes, considering compute, RAM, and power. Testing applications on representative hardware is crucial for understanding real-world performance and making informed placement decisions. The economics of local inference involve trade-offs between local compute costs and cloud inference expenses.