From Feature Engineering to Context Engineering

For years, customer intelligence progressed one output at a time. A churn score, a time-to-buy estimate, a segmentation, then a new pipeline whenever a new question appeared.

That architecture has one clear strength: each result can be defined, evaluated, and industrialized. It also has a cost. Models, features, and pipelines accumulate until the system becomes a catalog of answers chosen in advance.

Local LLMs create another option. They can make CRM notes, support tickets, transcripts, and product reviews useful close to the data, without turning every new question into a scoring project.

The important shift is not the disappearance of feature engineering. It is the arrival of a reusable context layer alongside structured models.

#One question, one model, one pipeline

Feature engineering turns stable data into variables a model can use. To estimate churn risk, a team might calculate purchase frequency, basket evolution, returns, or time since the last interaction.

This approach remains strong when the question is stable and repeated. The score can be monitored over time, compared with observed outcomes, and integrated into a campaign with clear rules.

The limit appears when questions change faster than pipelines. Why is a premium segment eroding? Which irritants recur in recent conversations? Which weak signals never make it into a customer profile column?

The two approaches do not produce the same object. One supplies a stable signal for a repeated decision. The other helps explore a situation whose causes are scattered across several sources.

#The messy parts become useful

Much of what an organization knows about a customer already exists in a form that is difficult to use. It sits inside free-text comments, conversations, and reports that traditional pipelines often leave aside.

Context engineering does not mean pouring all of that data into a prompt. The system still needs to select useful sources, enforce access rights, preserve provenance, and request an output that downstream users can work with. Context has to be engineered as carefully as a pipeline.

Only then do the messy parts become operational. They stop being dead weight inside the CRM and add evidence that structured attributes cannot capture on their own.

#Why local inference changes the boundary

This approach is becoming more credible because capable models can now run on infrastructure an organization controls. Local inference reduces data transfers and dependence on a third-party API. It does not make a system compliant or secure by itself.

Qwen3.5-35B-A3B has 35 billion total parameters but activates 3 billion per token. That MoE architecture reduces the computation performed at each step. It does not remove the memory required to store the model weights, although quantization can reduce it substantially.

DeepSeek-V3 pushes the principle much further, with 671 billion total parameters and 37 billion activated per token. It is an important signal about model architecture, not evidence that a model of this size suddenly fits on an ordinary machine.

The software layer is progressing at the same time. llama.cpp supports low-bit quantization, multiple hardware backends, and hybrid CPU+GPU inference. llama-swap adds on-demand model switching, OpenAI-compatible endpoints, and automatic unloading after a TTL.

These tools do not remove the trade-offs between quality, memory, and latency. They make those trade-offs manageable on simpler infrastructure, with several specialized models behind a common interface.

#The customer profile becomes a context layer

The customer profile is no longer limited to a collection of attributes calculated in advance. Under the right controls, it can provide the information needed to answer a new question.

Consider a CRM lead trying to understand why a premium segment is eroding:

1. The segment and access rights define the scope.
2. The system retrieves authorized notes, tickets, and transcripts.
3. The model extracts recurring signals and returns a structured answer with its sources.
4. The CRM lead validates the analysis before any decision or activation.

The next question can use the same infrastructure with a different context. This does not guarantee a good answer. It avoids rebuilding the entire chain before the team can even explore the problem.

Explicit models do not disappear. They become one layer of intelligence inside the system, not the entire system.

#Choosing the right production pattern

A structured score remains preferable for a frequent, measurable, and highly automated decision. A context layer becomes useful for exploratory questions, narrative sources, or needs that evolve quickly.

The choice is not between old and new. It depends on the nature of the decision, the required level of auditability, and the cost of an error. Both approaches still need authorized data, evaluation criteria, and a clearly identified owner.

The deeper shift is this: customer intelligence is no longer limited to attributes chosen when a pipeline was designed. It can also draw on governed context to answer questions that did not yet exist.

Local models make that evolution possible without making it simple. That is exactly why product, architecture, and governance need to move together.

If this shift reflects the questions you are asking about your CRM or CDP, you can reach me at octave@olivetti.ai.