TL;DR: From our experiences and chats with clients on deploying AI we see some distinct patterns emerging. One of those conversations was on Beyond the Hype: Is architecture for AI even necessary? With Russell who Heads up AI at Atom Bank – he even coined the term “Artisan AI” on that podcast which has been carried forward into our framework:

Please note this is not a maturity matrix! Multiple approaches can apply simultaneously – as different patterns work better for different workloads / challenges. Keep reading for more details.

Finding an appropriate approach to AI

Naturally before going into this we need to acknowledge that AI is not appropriate for every situation. Much like any new emerging technology there are many risks and considerations (as I touched on in my piece from 2023). I think this is partly why many organisations are seeing challenges with success and ROI from GenAI projects at the moment. It’s being used in a blanket fashion as many are chasing hype rather than genuine value. Anyway assuming you have concluded that Generative AI is appropriate and there is sufficient tangible value a crucial question emerges for enterprise architects: how might we responsibly implement AI while balancing innovation with governance?

So as I said at the start we can perhaps identify what looks like a spectrum of architectural approaches for enterprise AI deployment. This ranges from informal augmentation or “vibing” through to highly regulated, mission-critical deployments. Each approach appears to offer distinct trade-offs in terms of speed, control, risk and value.

In this article I am going to be exploring this spectrum with the goal of helping you consider where your organisation is today and where it might best position itself. Depending on the scale of your organisation, it may be that multiple approaches are being taken simultaneously across different programmes and initiatives – different patterns work better for different workloads/challenges.

The Spectrum of AI Architectural Approaches

“Augmentation” – Individual Ad-hoc Approaches aka “Vibing”

At what might be the most informal end of our spectrum lies what is termed “Augmentation” – the individual, ad-hoc use of off-the-shelf AI co-pilot type tools. In many cases, this is without much in the way of formal organisational oversight – in fact, this can often fall into the shadow AI or “bring your own AI” category.

Picture an analyst crafting prompts in ChatGPT or Claude.ai to help analyse survey data, or a developer using GitHub Copilot to accelerate coding tasks. These scenarios are characterised by some or all of:

• Individual users leveraging AI independently, much like how Excel spreadsheets have been used

• “Bring your own AI” situations where employees use personally selected or company-sanctioned AI tools. Individuals are taking it upon themselves to outsource or speed up their workflow through use of their own personal AI platforms – the risks that entails have been touched on in our piece about de-risking generative AI.

• Heavy reliance on individual expertise for effective prompting and output evaluation, with consequent inconsistency across team members (unless some cross team management of prompts is happening)

• Low barrier to entry but limited control over data privacy, IP leakage (unless open source AI is being locally – via the likes of LM Studio or GPT4ALL) and output quality

• Risk of being charmed by overconfident assistants – something like Replit might build you what it claims is a fully working application, only to discover it has serious flaws or missing features it claims it built.

This approach could enable exploration and innovation, but might present thorny challenges around consistency, security, and intellectual property protection. It’s where many organisations perhaps begin their AI journey before evolving toward more structured approaches.

Experimentation

These are often proof of concepts and pilots in highly regulated business or limited production testing in less regulated consumer use. The objective is to understand the art of the possible and feasibility check new approaches and technologies.

Experimentation generally goes beyond augmentation – typically with more bleeding edge models or frameworks including (at the time of writing!) **agentic & reasoning models **or novel architectures such as KnowPath.

With the right approach to architecture (think more thought through than throwaway vibe coding!) they can move towards artisanal or mainstream approaches that are discussed later on.

“Artisan AI” – Enterprise-Controlled AI Architecture

Whilst others might be “vibe coding” you can think of Artisan as the opposite approach. Artisan is a far more human engineered / controlled, and enterprise owned deliberate strategy:

• Similar in many respects to the “Mainstream” approach described later on but with models hosted within infrastructure the organisation controls

• Leverages open source models (like Llama, Mistral, or other models on model hubs like Hugging Face) deployed on enterprise-controlled infrastructure

• Often feature a “deterministic spine” (traditional programming logic) to control the AI powered workflow to manage risk and ensure consistency or experience and outcome

• Provides greater intellectual property protection and security control through self-hosting (or public cloud although this arguably removes some of the benefits – so more likely managed co-location or private cloud)

• So, looking at a range of on-device, on-premise, edge or private cloud hosted models for maximum control and flexibility. This distributed architecture can have environment, cost and go to market speed advantages too (if you are maxing use of existing assets in your ecosystem).

• Suitable for sensitive or highly regulated use cases where data sovereignty is paramount

• Enables fine-grained control over model selection, versioning and deployment within a managed enterprise “Model Zoo”. This takes external models from Model Gardens / Hubs and where required [in house] data science teams building enterprise specific models

• May include custom foundation open source models fine-tuned to organisational data and business domains

• Can involve extended and customised AI solutions through integration with domain-specific data almost certainly combined with RAG and/or GraphRAG / KnowPath type approaches

This approach offers the structured governance of mainstream implementations while maintaining full ownership of the majority of the technology stack, with the ability to eliminate dependency on external API providers and, for very sensitive workloads, run offline and air-gapped. The higher level of control makes this approach particularly suitable for regulated industries or sensitive applications. See the previous paper on de-risking AI for more details on architecture for highly regulated applications.

It mitigates data privacy concerns by ensuring sensitive data never leaves organisational boundaries, while providing the flexibility to deploy and manage multiple models based on specific use case requirements. In fact, anyone that cares enough about their intellectual property being incorporated into future AI products would be wise to consider this approach. Remember this isn’t just about training data, people are underestimating the value of prompts as the next layer of future training data or prompt engineering for creating higher value vertical-specific AI products. After all if you own the architecture housing the AI, you own all of that valuable data. Not enough thought is being given to the value of the data given away at inference – it potentially reveals valuable thought patterns, brand value or processes. Whilst assurances are being made that prompts aren’t being used it is somewhat hard to fully trust the AI providers. Many of them have had a somewhat grey zone approach to IP for the training phase and they are increasingly under pressure to monetize their expensive operations. They could probably find a loophole like using meta data or higher level patterns of usage rather than individual prompts (after all we are already seeing analysis of what people are using LLMs for i.e. https://www.anthropic.com/news/the-anthropic-economic-index). What will the AI model companies do in the pursuit of profit and/or future product dominance?

Augmented SaaS – AI Add-ons and Enhancements

The next stage in our spectrum (the axis has been going individual use towards mass use – and arguably this category could be more widely used than mainstream depending on your organisation) involves the integration of AI capabilities into existing enterprise software:

• Existing enterprise SaaS platforms (Productivity, CRM, ERP, low-code tools) incorporating AI features

• Team-level and enterprise-wide usage rather than purely individual adoption

• What Gartner might call the “Consume” approach – using GenAI embedded in applications

• Potential privacy concerns about enterprise data usage with somewhat opaque AI functionality

This approach seems to be gaining popularity as organisations explore ways to leverage AI capabilities without significant architectural changes to existing systems. The governance challenge here might involve managing the consistent and appropriate use of these embedded capabilities across the enterprise. There are also challenges as vendors decide to implement AI behind the scenes on existing SaaS applications without considering what that does to customer risk profiles or appetite to adopt platform AI. This was something that Russell raised on the Beyond the Hype episode I mentioned earlier - Beyond the Hype: Is architecture for AI even necessary?

“Mainstream” – API-Based Architectural Integration

This has typically been what most organisations have been doing for enterprise AI applications. Using a perceived safe brand (like OpenAI or Microsoft) for API access to a LLM that can be embedded within an enterprise use case (which itself is probably also hosted on a public cloud provider). So it’s structured approaches to integrating AI via APIs:

• Also referred to as the “Embed” approach identified by Gartner – integrating GenAI APIs into custom application frameworks

• Enterprise-grade internal- and external-facing processes powered by a mix of LLM controlled and deterministic workflows (lower risk traditionally programming logic) with specific AI usage

• Use of cloud-hosted, off-the-shelf models accessed through APIs

• Potential concentration, dependency, intellectual property, sustainability, cost (many of these services are currently priced below a sustainable break even point by vendors to try and achieve usage and market share) and geopolitical risks

• Often extended with techniques like Retrieval Augmented Generation (RAG) to incorporate enterprise data

• May involve custom data pipelines to incorporate organisational context while using external AI capabilities

This approach could offer greater control than pure SaaS consumption while avoiding the complexity of custom development. It seems to be becoming a common pattern for organisations wanting to move beyond experimentation while managing risk.

But have you considered the potential concentration risks? What might happen if your chosen API provider doubles their prices or goes out of business? Have you built in the architectural flexibility to switch providers if needed?

Data: The Foundation That Cannot Be Overlooked

Beyond the architectural deployment style underpinning all AI deployment is data. If there’s one consistent story that we can see from enterprise AI implementations, it’s that data is the crucial foundation for success:

• 75% of organisations are increasing investments in data management specifically for AI implementation, according to Deloitte’s AI Institute “State of Generative AI in the Enterprise” report

• The same report indicates that data-related issues have caused 55% of surveyed organisations to avoid certain AI use cases entirely

• Our experience on client and R&D projects mirrors this. The effectiveness of InferESG (a tool we developed using agentic AI) massively increased with the addition of ESG data sets and documents that establish materiality and provide expert grounding.

• Quality, accessibility, and governance of data could directly impact the effectiveness of AI systems regardless of architectural approach

Sustainable AI Architecture: A Call to Action

Many organisations sadly seem to be pursuing AI with little regard for long-term sustainability. Sometimes this is due to lack of awareness, other times it’s just plain short sightedness. This could create significant risks:

Financial Sustainability

The true cost of AI goes beyond initial implementation. Have you modelled the total cost of ownership, including ongoing inference costs, model updates, and potential vendor lock-in? What happens if (and we have already started to see this with the cost of Microsoft Co-pilot) the tech platforms increase their prices to cover costs? Or look to monetise your data or AI product experience in different ways? You should bake likely cost increases into your Total Cost of Ownership (TCO) models – unless we start to see drastic improvements to model efficiency or deployment architectures from the main AI vendors.

Environmental Sustainability

The environment footprint (carbon, materials, water & nature impacts) of large AI models is substantial – they require massive amounts of hardware and energy for training and inference (running) the underlying platforms. Are we considering the environmental impact of our architectural choices? These could at scale start to threaten corporate net zero or sustainability commitments. Edge computing and smaller, more efficient models could dramatically reduce inference energy consumption and the embodied carbon from the training phase.

Operational Sustainability

Do you have the skills and processes to maintain increasingly complex AI systems? How will you manage the inevitable technical debt that comes with rapidly evolving technology? The speed of change, rate of adoption and sheer volume of content being generated by AI is going to create a lot of noise and future technical debt to deal with. In a lot of cases AI is just being slapped on top of already messy and complex architectures. So instead of lowering cost and making thing simpler it may well come back to bite.

Questions to Consider

As you navigate your organisation’s AI journey, consider these critical questions:

1. Risk Assessment: Where on the spectrum should different AI use cases in your organisation sit, based on risk profile and value potential? Is there a danger you are force fitting an AI solution that isn’t quite good enough that might create reputation or regulatory risks from a degraded experience?

2. Governance Framework: Have you established clear policies for AI development, usage, and monitoring that align with your organisational values and regulatory requirements? Just because AI is new, shiny and exciting doesn’t mean it should get to ride roughshod over sensible controls. As Russell said on the podcast – engage early with security and risk stakeholders.

3. Data Readiness: Is your data architecture mature enough to support your AI ambitions? What investments are needed to establish the necessary foundation?

4. Skills Gap Analysis: Do you have the technical capabilities to implement and maintain your chosen architectural approach? If not, what’s your plan to address this? Does the average person involved

5. Concentration Risk: How dependent will your architecture make you on specific vendors or technologies? What’s your mitigation strategy?

6. Change Management: How will you manage the organisational change required to effectively integrate AI into business processes? How are capabilities and end to end processes being re-imagined and optimised (with an external ecosystem view not just an internal view – after all what will clients and stakeholders want / expect?)

7. Ethical Considerations: What safeguards have you put in place to ensure your AI implementations align with ethical principles and avoid bias?

Striving for a Pragmatic Path

The path to effective enterprise AI architecture is rarely linear. Most organisations will likely maintain multiple approaches simultaneously for different use cases, gradually progressing towards more mature implementations as their AI capability develops.

The key to success will partly lie in matching your architectural approach to both the use case requirements and your organisational maturity. An experimental lab might be appropriate for exploring new internal non-critical capabilities, while customer-facing applications might demand the control of a custom deployment.

Rather than a one-size-fits-all or overly burdensome governance approach, consider an AI architectural decision framework that guides teams toward the appropriate implementation pattern based on data sensitivity, regulatory requirements, performance needs, and business criticality.

Conclusion: Architecture and considered trade-off are vital

AI is making architecture more critical; most new applications have been following increasingly standardised cloud platforms, and this certainly shakes things up. It is a time to differentiate and innovate with architecture to manage the risks to your organisation whilst allowing for a whole new wave of innovation.

The bottom line is that you need to work with your risk, security, regulatory and/or legal stakeholders as early as possible. Collaborate with them so they understand the unique characteristics of the AI models you want to work with. Treat the associated non-functional requirements and quality attributes as first-class citizens. This is an area for innovation that generally gets overlooked but is vital for safe and successful enterprise adoption of groundbreaking models.

As we navigate this latest phase of tech, the organisations that thrive will be those that balance innovation with governance, move beyond the hype to practical implementation, and build sustainable architectural approaches that create lasting value. If you’d like a chat more about this feel free to find me on LinkedIn or email me at oliver@scottlogic.com