Are teams still using Pytorch/Tensorflow, or is most ML work just calling LLM endpoints and prompt engineering now?

In today’s hiring landscape, the shift toward LLM‑centric roles is unmistakable. A growing number of job postings focus on prompt engineering, retrieval‑augmented generation, and integrating LLM endpoints with frameworks like LangChain, rather than on training models from scratch. This trend mirrors a broader industry move: the “model‑as‑a‑service” paradigm has become the default path for most organizations looking to embed AI into their products quickly. If you’re exploring new opportunities, it pays to understand why this shift has happened and what it means for your career trajectory. For context, see [D]Trying to switch back to AI/ML — what skills are actually in demand right now?[R] and the broader discussion in our feature on AI/ML career transitions. These pieces outline how foundational knowledge still matters, but the tools and frameworks you master will dictate your employability.

The decline of in‑house PyTorch and TensorFlow projects is not a disappearance of core machine learning skills; it is a re‑prioritization. Training a large neural network from scratch is resource‑intensive, requiring significant compute budgets, data pipelines, and continuous monitoring. For many businesses, the return on investment from building a custom model is outweighed by the speed and reliability of an established LLM provider. By calling an external endpoint, teams can focus on the unique business logic—such as domain‑specific retrieval or custom prompt design—while outsourcing the heavy lifting of model inference. This shift also reduces the barrier to entry for smaller teams: instead of maintaining GPU clusters and tuning hyperparameters, they can deploy an LLM with a few lines of code and iterate rapidly on the user experience.

However, this doesn’t render PyTorch or TensorFlow obsolete. These frameworks remain crucial for research labs, academic projects, and industries with strict data privacy or regulatory constraints that demand on‑premise solutions. Moreover, the skill set of a practitioner who can navigate both worlds—understanding how to fine‑tune a model and how to integrate it into a production pipeline—remains highly prized. Companies that need to balance cost, compliance, and innovation often adopt a hybrid approach: they use LLM endpoints for general tasks but build custom models for niche applications where data sensitivity or performance gaps exist. For engineers, this means a future where versatility trumps specialization: the ability to engineer prompt strategies, manage data ingestion pipelines, and orchestrate end‑to‑end ML workflows will set you apart.

Looking ahead, the conversation will pivot from “what tools do I need?” to “how do I design AI systems that are ethical, explainable, and maintainable?” As LLM usage scales, so does the need for robust governance frameworks and transparent decision‑making. Companies are investing in tools that audit prompts, track model outputs, and ensure compliance with emerging regulations. For professionals, mastering these governance layers—alongside technical fluency—will be the new differentiator. So, while the day of building models from scratch is still alive, the market is increasingly rewarding those who can orchestrate the entire AI lifecycle, from data prep and prompt crafting to deployment and monitoring. The question for the next wave of talent is: are you ready to become the bridge between cutting‑edge LLM capabilities and the real‑world problems your organization faces?