Large Language Models for Enterprise: Fine-Tuning and Deploying Custom AI

Fine-tuning a large language model (LLM) for enterprise use means taking a pre-trained model (like Llama 3, Mistral, or GPT-4) and training it on your company’s specific data to improve accuracy for your use case. For Indian enterprises, the decision between fine-tuning an open-source model, using API-based models with prompt engineering, or building a custom model depends on data sensitivity, volume, and budget — with fine-tuning costs ranging from ₹50,000 for a small Llama adapter to ₹50+ lakh for full fine-tuning of a 70B+ parameter model.

Last updated October 2026

When Does an Enterprise Need a Custom LLM?

Most businesses don’t need fine-tuning. Start by asking these questions:

• Can prompt engineering solve it? For many tasks (summarisation, translation, Q&A, content generation), a well-crafted system prompt with few-shot examples using GPT-4, Claude, or Gemini via API achieves 80–90% of fine-tuned performance
• Is RAG (Retrieval-Augmented Generation) sufficient? If the model needs your company’s knowledge but the task is standard (customer support, document search), RAG + base model often outperforms fine-tuning
• Does the model need to learn a specific style, format, or domain language? Fine-tuning excels when you need consistent output formatting, domain-specific terminology (legal, medical, financial), or brand voice
• Is data privacy non-negotiable? If you cannot send data to external APIs (banking, healthcare, defence), self-hosted fine-tuned models are the answer

The Build vs Buy Decision for Indian Enterprises

Choosing the Right Base Model for Fine-Tuning

As of October 2026, the best open-source models for enterprise fine-tuning are:

• Llama 3.1 (Meta, 8B/70B/405B): Excellent general-purpose model, strong at code and reasoning. Commercially licensed for most uses. Best choice for most Indian enterprises
• Mistral Large / Mixtral (Mistral AI): Strong performance-per-parameter ratio, good for cost-sensitive deployments
• Qwen 2.5 (Alibaba): Excellent multilingual capabilities including Hindi and other Indic languages. Consider for Indian language tasks
• Gemma 2 (Google): Lightweight (2B/9B/27B), good for edge deployment on limited GPU hardware

For Hindi and regional Indian language support, test Qwen 2.5 and Llama 3.1 — both have reasonable Indic language understanding, though neither matches their English performance.

Fine-Tuning Methods Explained

Full Fine-Tuning

Updates all model parameters. Requires significant GPU memory — a 70B model needs 8x A100 80GB GPUs minimum. Cost: ₹5–50 lakh for training depending on dataset size and compute hours. Rarely necessary for most enterprise use cases.

LoRA / QLoRA (Recommended for Most Cases)

Low-Rank Adaptation adds small trainable adapter layers while keeping the base model frozen. Benefits:

• 70B model fine-tuning on a single A100 GPU (with QLoRA quantisation)
• Training cost: ₹50,000–₹3 lakh depending on dataset and epochs
• Multiple task-specific adapters can share one base model, reducing deployment costs
• 90–95% of full fine-tuning performance for most tasks

RLHF / DPO (Preference Alignment)

Used after supervised fine-tuning to align the model with human preferences. Important for customer-facing applications where tone, safety, and helpfulness matter. Requires a preference dataset (human-ranked response pairs), which can cost ₹2–10 lakh to create for Indian-specific use cases.

Data Preparation for Fine-Tuning

Data quality determines fine-tuning success more than model size or training budget. For Indian enterprises:

1. Collect diverse examples: 1,000–10,000 high-quality input-output pairs for LoRA; 50,000+ for full fine-tuning
2. Clean and deduplicate: Remove personal data (DPDP Act compliance), standardise formatting, check for biases
3. Handle multilingual data: If your users switch between English and Hindi mid-sentence (common in Indian business communication), include code-mixed examples
4. Create evaluation sets: Reserve 10–20% of data for testing. Define clear metrics: accuracy, F1 score, or task-specific measures
5. Version control your data: Use DVC (Data Version Control) or similar tools. Data changes affect model behaviour just like code changes

Infrastructure and Hosting in India

Where to run fine-tuning and inference for Indian enterprises:

• AWS Mumbai (ap-south-1): p4d.24xlarge (8x A100) at ~₹2,500/hour. Most mature GPU offering in India
• Azure Central India: NC-series VMs with A100/H100 GPUs. Good integration with Azure AI Studio
• GCP Mumbai: Limited GPU availability compared to AWS. Better for Vertex AI managed services
• Indian GPU cloud (E2E Networks, Yotta): Competitive pricing for long-term reservations, data residency guarantee. E2E offers A100 instances from ₹1,000–₹1,500/hour
• Inference optimisation: Use vLLM or TensorRT-LLM for 2–4x throughput improvement. Quantise to 4-bit or 8-bit for inference (GPTQ, AWQ) to reduce GPU requirements

DPDP Act Compliance for Enterprise AI

India’s Digital Personal Data Protection Act 2023 impacts LLM deployment:

• Training data: If your fine-tuning data contains personal information, you need a lawful basis for processing under DPDP
• Data localization: The Act allows cross-border transfer except to government-blacklisted countries. However, many Indian enterprises prefer data residency in India for risk management
• Right to erasure: If a user requests data deletion, you may need to demonstrate that their data was removed from training sets (technically challenging with LLMs)
• Purpose limitation: AI models trained on customer support data cannot be repurposed for marketing without fresh consent

Real-World Use Cases for Indian Enterprises

• Legal document analysis: Fine-tuned models understanding Indian contract law, Companies Act, and GST regulations. Law firms like Cyril Amarchand and Khaitan & Co are investing in this
• Multilingual customer support: Models handling Hindi-English code-switching for D2C brands, banking, and telecom
• Financial report generation: Automated quarterly/annual report drafting for listed companies using SEBI-compliant formatting
• Agriculture advisory: Models trained on Indian crop data, soil types, and MSP information for agri-tech platforms
• Healthcare triage: Models understanding Indian medical terminology and regional disease patterns for telemedicine platforms

Frequently Asked Questions

How much does it cost to fine-tune an LLM for an Indian enterprise?

For LoRA/QLoRA fine-tuning (recommended): ₹50,000–₹3 lakh for training, plus ₹30,000–₹2 lakh/month for GPU inference hosting. Full fine-tuning of a 70B model: ₹5–50 lakh for training alone. Include ₹2–10 lakh for data preparation and evaluation.

Can I fine-tune models on Indian languages like Hindi, Tamil, or Bengali?

Yes, but with caveats. Llama 3.1 and Qwen 2.5 have decent Hindi support. For Tamil, Bengali, Telugu, and other Indic languages, you’ll need more fine-tuning data to compensate for lower base model proficiency. AI4Bharat’s IndicLLM and Sarvam AI’s models are worth evaluating for Indic-first applications.

Is it better to use OpenAI’s API or fine-tune an open-source model?

Start with OpenAI/Anthropic API + prompt engineering. If you need: (a) data to stay in India, (b) cost reduction at scale (>100K+ API calls/month), (c) specialised domain performance, or (d) offline/air-gapped deployment — then fine-tune an open-source model. The crossover point where self-hosted becomes cheaper is typically ₹2–3 lakh/month in API costs.

What team do I need for an enterprise LLM project?

Minimum viable team: 1 ML engineer (fine-tuning, evaluation), 1 backend engineer (API, infrastructure), 1 domain expert (data quality, use case definition). Timeline: 2–4 months from concept to production. For Indian hiring, expect ₹15–40 lakh/year per ML engineer with LLM experience.

How do I evaluate if my fine-tuned model is actually better than the base model?

Create a benchmark dataset of 200–500 examples representing real production queries. Measure: task accuracy, response quality (human evaluation), latency, and cost per query. A/B test the fine-tuned model against the base model + prompt engineering on real user traffic. If the fine-tuned model doesn’t show at least 10–15% improvement on your key metric, prompt engineering may be sufficient.