Back to Blog

How to Connect AI to Your Existing Business Systems

April 20, 2026•11 min read•Michael Ridland

You've built (or bought) an AI system. It works brilliantly in isolation. Now you need it to actually talk to your CRM, your ERP, your accounting system, and that legacy database running on a server in the corner that nobody wants to touch.

Welcome to the part of AI that nobody puts in their demo video.

In our experience at Team 400, integration work accounts for 50-70% of the total effort in most enterprise AI projects. The AI model itself is often the easy part. Getting it to reliably read and write to your existing business systems - that's where projects succeed or fail.

Here's how to approach it.

Why AI Integration Is Harder Than Regular Integration

You might think connecting AI to your systems is just another API integration. It's not, and here's why.

AI requests are unpredictable. A traditional integration has a defined set of operations - create order, update customer, fetch invoice. An AI system may need to call any combination of operations in any order depending on the conversation or task. You can't pre-define every flow.

AI needs context across systems. A customer service AI doesn't just need your CRM. It needs CRM data, order history from the ERP, shipping status from the logistics system, and support history from the ticketing platform. All in one interaction. All fast.

AI makes mistakes. Traditional integrations either work or they don't. AI systems can call the right API with the wrong parameters because they misunderstood the user's intent. You need safeguards that traditional integrations don't require.

AI operates at conversation speed. Users expect responses in seconds. If your AI needs to call four systems and each takes 3 seconds, you've lost the user's patience.

The Architecture That Works

After dozens of enterprise AI integration projects, we've settled on an architecture that reliably handles the complexity.

The Tool Gateway Pattern

Never let your AI system call external APIs directly. Always route through a gateway layer.

User -> AI Agent -> Tool Gateway -> Your Business Systems
                        |
                        +-- Authentication
                        +-- Input Validation
                        +-- Rate Limiting
                        +-- Logging & Audit
                        +-- Error Handling
                        +-- Circuit Breakers

The tool gateway is a thin API layer that your AI calls. It handles all the messy reality of enterprise systems - authentication, retries, error handling, logging - so the AI layer stays clean.

Benefits we've seen in practice:

  • Security: Credentials never touch the AI layer. The gateway handles all authentication with external systems.
  • Control: You can rate-limit, throttle, or disable any integration without changing the AI system.
  • Visibility: Every call to every system is logged in one place. When something goes wrong (and it will), you can trace exactly what happened.
  • Testing: You can mock the gateway layer for testing without needing live connections to production systems.

Defining Tools for the AI

Your AI system interacts with the gateway through a defined set of "tools" - structured descriptions of what operations are available, what parameters they accept, and what they return.

Here's what a well-defined tool looks like:

{
  "name": "get_customer_orders",
  "description": "Retrieves recent orders for a customer. Use when the user asks about order status, delivery, or purchase history.",
  "parameters": {
    "customer_id": "string (required) - The customer's account ID",
    "limit": "integer (optional, default 10) - Number of orders to return",
    "status_filter": "string (optional) - Filter by order status: pending, shipped, delivered, cancelled"
  },
  "returns": "List of orders with order_id, date, status, items, and total amount"
}

The description matters more than you think. The AI uses it to decide when to call this tool. Vague descriptions lead to wrong tool selections.

In our experience, spending time on clear, specific tool descriptions reduces AI errors more than any amount of prompt engineering.

Read vs Write Separation

Treat read operations and write operations completely differently.

Read operations are low-risk. The AI fetches data, processes it, and returns information to the user. If the AI calls the wrong read endpoint, the worst case is showing irrelevant information.

Write operations are high-risk. The AI creates an order, updates a customer record, sends an email. Getting this wrong has real consequences.

For write operations, we always implement:

  • Confirmation prompts: The AI shows what it intends to do and asks for confirmation before executing
  • Validation rules: The gateway validates all parameters before executing the write
  • Rollback capability: Where possible, write operations can be undone
  • Audit logging: Every write operation is logged with full context (who, what, when, why)

Connecting to Common Business Systems

CRM Integration (Salesforce, HubSpot, Dynamics 365)

CRMs are usually the first system to connect because customer context is valuable across almost every AI use case.

What works well:

  • Fetching customer profiles and interaction history
  • Looking up account details and relationships
  • Logging AI interactions as activities
  • Creating follow-up tasks

Watch out for:

  • API rate limits: Salesforce in particular has strict API limits. Cache frequently accessed data. Batch where possible.
  • Field-level security: Your API credentials may see fields the user shouldn't. Implement filtering in the gateway.
  • Complex object models: CRM data models are often customised heavily. Map out the actual data model, not the standard one.

Typical approach:

  • OAuth 2.0 for authentication
  • REST API for individual operations
  • Bulk API for data synchronisation
  • Webhooks for real-time updates from CRM to AI

ERP Integration (SAP, Oracle, NetSuite, MYOB)

ERP integration is where things get complicated.

What works well:

  • Order status lookups
  • Inventory availability checks
  • Pricing and quoting
  • Customer credit status

Watch out for:

  • Middleware is usually required: SAP and Oracle rarely expose clean APIs. You'll need an integration layer (Azure Integration Services, MuleSoft, or custom middleware).
  • Transaction integrity: ERPs manage complex transactions. An AI that partially updates an order can cause serious problems. Use the ERP's built-in transaction handling.
  • Performance: ERP queries can be slow. Use read replicas or cached views for AI-facing operations.
  • Master data management: ERPs are often the system of record. Be very careful about AI writing back to master data.

Legacy Databases and Systems

Every organisation has them. The system that's been running for 15 years with no documentation and one person who understands it.

Options in order of preference:

  1. Build an API wrapper: Create a lightweight API that sits in front of the legacy database. The API encapsulates the business logic and data access. This is the most maintainable option.

  2. Read-only database access: Connect to the legacy database for reads only. Use views to expose the data in a clean format. Never let the AI write directly to a legacy database.

  3. File-based integration: Export data from the legacy system on a schedule. Import it into a system the AI can query. Works for data that doesn't need to be real-time.

  4. RPA as a bridge: Use robotic process automation to interact with the legacy system's UI. This is the option of last resort - it's fragile, slow, and maintenance-heavy. But sometimes it's the only option.

Email and Communication Systems

AI systems often need to send emails, create calendar events, or post messages.

Microsoft 365 / Graph API:

  • Well-documented API
  • Granular permissions (good for security)
  • Supports mail, calendar, Teams, and OneDrive
  • Webhook support for incoming messages

Google Workspace:

  • Similar capabilities via Google APIs
  • Different authentication model (service accounts)

Key consideration: When AI sends communications on behalf of users, make it clear the message was AI-generated. This is both a trust issue and increasingly a regulatory requirement.

Handling Authentication Securely

This is non-negotiable. Get authentication wrong and you've created a security disaster.

Rules:

  1. Never put credentials in prompts or AI context. The AI layer should have zero knowledge of passwords, API keys, or tokens.

  2. Use a credential store. Azure Key Vault, AWS Secrets Manager, or HashiCorp Vault. The gateway retrieves credentials at runtime.

  3. Use managed identities where possible. On Azure, managed identities eliminate the need for credentials entirely for Azure-to-Azure connections.

  4. Implement per-user context. The AI should access external systems with the permissions of the user it's serving, not with a superuser account. This means passing user context through to the gateway and using it for authentication.

  5. Rotate credentials regularly. Automated rotation through your credential store.

AI Agent -> Tool Gateway -> Azure Key Vault -> External System
                                    |
                           (Credentials retrieved
                            at runtime, never stored
                            in application code)

Performance - Making It Fast Enough

Users expect AI responses in 2-5 seconds. If your AI needs to call three systems and each takes 2 seconds sequentially, you're already over budget.

Run Calls in Parallel

If the AI needs data from multiple systems, fetch it all at once.

# Sequential - 6 seconds total
customer = await crm.get_customer(id)
orders = await erp.get_orders(id)
tickets = await support.get_tickets(id)

# Parallel - 2 seconds total (slowest call)
customer, orders, tickets = await asyncio.gather(
    crm.get_customer(id),
    erp.get_orders(id),
    support.get_tickets(id)
)

Cache Reference Data

Data that changes slowly should be cached:

  • Product catalogues (refresh hourly)
  • Pricing tables (refresh when updated)
  • Organisation structures (refresh daily)
  • Policy documents (refresh when published)

Use event-driven cache invalidation where possible. Time-based expiry as a fallback.

Pre-fetch Common Patterns

If you know that every customer service interaction starts with customer lookup + recent orders, pre-fetch that data when the conversation starts rather than waiting for the AI to request it.

Set Timeouts Aggressively

Every external call should have a timeout. If a system doesn't respond in 5 seconds, return a graceful degradation rather than making the user wait.

"I can see your account details but I'm having trouble loading your recent orders right now. Let me help with what I have, or I can try again in a moment."

Error Handling That Doesn't Break the Experience

External systems fail. Networks have issues. APIs return unexpected responses. Your AI integration needs to handle all of this gracefully.

The Circuit Breaker Pattern

If an external system starts failing, stop calling it rather than flooding it with requests that will also fail.

Track success and failure rates for each integration. If the failure rate exceeds a threshold (say, 50% of calls in the last minute), open the circuit breaker - stop making calls and return a fallback response.

After a cooling period, try again. If it works, close the circuit. If not, keep it open.

Graceful Degradation

Your AI should work (in a reduced capacity) even when some systems are down.

  • CRM down? The AI can still answer general questions using its knowledge base.
  • ERP down? The AI can acknowledge the order query and offer to follow up when the system is available.
  • Email system down? Queue the message and send when it recovers.

Never show raw error messages to users. "Error 503: Service Unavailable" is not a customer experience.

Retry with Backoff

For transient failures, retry with exponential backoff:

  • First retry: 1 second delay
  • Second retry: 2 seconds
  • Third retry: 4 seconds
  • Then give up and use graceful degradation

Testing Your AI Integration

Integration Tests

Test each connection point independently:

  • Can you authenticate?
  • Can you read data?
  • Can you write data?
  • What happens when the system returns an error?
  • What happens when the system is slow?
  • What happens when the data format is unexpected?

End-to-End Tests

Test complete AI flows with real (or realistic) system connections:

  • Full customer service conversation touching CRM + ERP + ticketing
  • Document processing flow from intake to storage to notification
  • Scheduling flow checking calendars, sending invites, handling conflicts

Load Tests

AI systems can generate more API calls than you expect. A busy AI agent might make 10-20 external calls per conversation. At 100 concurrent conversations, that's 1,000-2,000 API calls per minute.

Test your integrations under realistic load. Find out where the bottlenecks are before your users do.

Common Mistakes We See

Giving the AI too much access. Start with the minimum set of tools and expand as needed. An AI with access to everything is both a security risk and more likely to make errors (more options = more ways to go wrong).

Not logging enough. When an AI integration fails in production, you need to reconstruct exactly what happened. Log every tool call with full parameters and responses.

Treating all integrations the same. A read from a CRM and a financial transaction need fundamentally different safeguards. Tier your integrations by risk.

Ignoring data quality. AI integration exposes data quality problems you didn't know you had. Prepare for a data cleanup phase early in the project.

Building everything custom. For common systems like Salesforce, HubSpot, or Microsoft 365, there are well-tested integration libraries and connectors. Use them. Save your custom development budget for genuinely unique integrations.

Where Team 400 Can Help

Integration is one of the most common areas where AI projects stall. We've connected AI systems to CRMs, ERPs, legacy databases, document management systems, and custom internal applications across Australian businesses.

Our AI integration services team handles the architecture, security, and plumbing so the AI layer can focus on delivering value. We work with Azure AI Foundry and modern integration patterns to build connections that are reliable, secure, and maintainable.

If you're planning an AI deployment and worried about the integration challenge, talk to our team. We'll give you an honest assessment of what's involved and where the hidden complexity is.