ε Pulse: Issue #16

AI Scientists 🧑🏻‍🔬, AI Engineers 👨‍💻, and Why Prompt Engineering is Dead ⚰.

Unrelated to AI: 

• See my latest Around the Web newsletter. 

• Watch my latest video essay.

---

In this newsletter:

• Wet Lab Innovations will Lead the AI Revolution in Biology

• Extracting Human Interpretable Structure-Property Relationships in Chemistry Using LLM.

• AI-made Scientists for Scientific Discovery from Gene Expression Data.

• The Rise of AI Engineers

• Open Challenges for AI Engineering

• Parameter Efficient Fine Tuning with LoRa

• LangGraph Cloud and LangGraph Engineer

• Prompt Engineering is Dead

• There will be more AI agents than people

• Getting AI to do the Work of 700 people

• AI is a Services Revolution

• Generative AI: a New Consumer Operating System

• Meta LLM Open Source Projects, and more

[LLM in Science et al] 🤖 🦠 🧬

[I]: 👨‍🔬 Wet Lab Innovations will Lead the AI Revolution in Biology

This essay argues that the next significant breakthrough in AI for biology will come from wet-lab innovations rather than purely computational approaches. The author contends that while computational methods have yielded important advances like AlphaFold2, future progress will require new modalities of biological data that can only be generated through novel wet-lab techniques, particularly those that can collect data at scale.

“...But I also think many people have deeply misunderstood the ‘moral of the story’ of Alphafold2. The real takeaway was ‘ML can be extremely helpful in understanding biology’. But I worry that many people’s takeaway was actually ‘ML is singularly important in pushing biology forwards’. I don’t think this is true at all! In my opinion, what Alphafold2 pulled off — applying a clever model to a large body of pre-existing data to revolutionize a field — is something that will be extremely hard to replicate. Why? Because we’re almost out of that pre-existing data. If we had enough, sure, throw ML at it and call it a day, just as Alphafold2 did. But we don’t have that luxury anymore.” 

“We need new modalities of data to train our models with. Ideally, modalities that, 1, have complex underlying distributions, 2, are highly connected to physiologically important phenomena, and 3, are amenable to being collected at scale. Unfortunately, the data types that meet all of these 3 requirements have already been mined to death: protein sequences, protein structures, genomes, and transcriptomes.” 

[II]: 🧪 XpertAI: Extracting Human Interpretable Structure-Property Relationships in Chemistry Using XAI and Large Language Models

This paper introduces XpertAI, a framework that combines explainable AI (XAI) methods with LLMs to generate interpretable natural language explanations of structure-property relationships in chemistry from raw data

XpertAI takes as input a dataset containing molecular structures and target properties, along with relevant scientific literature. The framework first trains a surrogate machine learning model, such as XGBoost, on the input data to map molecular features to the target property. Explainable AI methods like SHAP or LIME are then applied to this model to identify the most important features influencing the prediction. A large language model such as GPT-4 is used to generate natural language explanations (NLEs) of the structure-property relationships by combining the results of the XAI analysis with domain knowledge extracted from the provided scientific literature. The final output is an interpretable textual explanation describing how specific molecular features relate to and influence the target property. These explanations include scientific justifications and literature references to support the proposed structure-property relationships. By combining the strengths of XAI and LLMs in this way, XpertAI addresses the challenge of making XAI results in chemistry more accessible and understandable to non-expert users, while maintaining scientific rigor by grounding the explanations in published literature evidence. This approach fills an important gap in extracting meaningful and trustworthy structure-property insights from raw chemical data.

[III]: 🧬Toward a Team of AI-made Scientists for Scientific Discovery from Gene Expression Data.

Summary: This paper introduces a Team of AI-made Scientists (TAIS) framework that uses LLM to automate the process of analyzing gene expression data for scientific discovery, focusing on identifying disease-predictive genes while considering various conditions.

Some details: 

The TAIS framework utilizes multiple AI agents to simulate the tasks typically performed by data scientists in analyzing gene expression data for scientific discovery. The system takes as input gene expression datasets from sources like TCGA and GEO, along with clinical information. The modeling approach involves several steps: First, a project manager agent decomposes the overall task and assigns subtasks to specialized agents. A data engineer agent, assisted by a domain expert, preprocesses the raw datasets. This involves cleaning the data, handling missing values, and extracting relevant features. Next, a statistician agent performs regression analysis to identify genes predictive of disease status while considering various conditions. The system employs techniques such as Lasso regression for variable selection in high-dimensional data, linear mixed models for confounding factor correction, and a two-step regression process for estimating missing conditions when necessary. Throughout the process, a code reviewer agent ensures the quality and correctness of the generated code. 

The output of the system includes a list of identified disease-predictive genes, their coefficients, and the prediction accuracy of the model. This framework fills a gap in automating complex scientific data analysis tasks in genomics by LLMs to replicate the collaborative efforts of a research team, potentially accelerating the pace of scientific discovery in this field.

Unfortunately, I couldn’t spot their code so I don’t know what’s going on behind the hood, but I suppose they are leveraging frameworks like LangGraph to orchestrate their AI scientists. 

[AI Engineering] 🤖🖥⚙

[I]: 👷The Rise of AI Engineers

I came across this essay a few months ago, and I really enjoyed reading it because it articulates what I have been noticing over the past 2 years or so - The rise of AI engineers. 

“We are observing a once in a generation “shift right” of applied AI, fueled by the emergent capabilities and open source/API availability of Foundation Models.

A wide range of AI tasks that used to take 5 years and a research team to accomplish in 2013, now just require API docs and a spare afternoon in 2023

…

In the near future, nobody will recommend starting in AI Engineering by reading Attention is All You Need, just like you do not start driving by reading the schematics for the Ford Model T. Sure, understanding fundamentals and history is always helpful, and does help you find ideas and efficiency/capability gains that are not yet in common consciousness. But sometimes you can just use products and learn their qualities through experience.

…

Microsoft, Google, Meta, and the large Foundation Model labs have cornered scarce research talent to essentially deliver “AI Research as a Service” APIs. You can’t hire them, but you can rent them — if you have software engineers on the other end who know how to work with them. There are ~5000 LLM researchers in the world, but ~50m software engineers. Supply constraints dictate that an “in-between” class of AI Engineers will rise to meet demand”

The authors also had an interesting podcast on How to Hire AI Engineers about a year after writing the essay

[II]: 🔖Open Challenges for AI Engineering

This talk is about the challenges and opportunities in LLM especially concerning the recent democratization of access to these models. The speaker, Simon Willson, discusses the rapid advancement in the capabilities of LLMs. Just a year ago, OpenAI’s GPT-4 was the best available model, but now there are several other options that are comparable or even better. This is largely due to the open-sourcing of some LLMs, including the release of Anthropic’s Claude series.

Willson emphasizes that while these powerful tools are becoming more accessible, there are important challenges to address. One challenge is ensuring that users understand the limitations of LLMs. LLMs are easily manipulated by prompts and can generate nonsensical or even harmful content if not used carefully. He terms this unreviewed and unrequested AI-generated content “slop” and argues that it is similar to spam. Another challenge Willson identifies is the lack of trust in AI systems. He mentions how recent controversies surrounding data privacy have eroded trust and argues that LLMs need to be developed and implemented in a way that respects user privacy. Amongst many others

[III] 👨‍💻Parameter Efficient Fine Tuning with LoRa

I am currently writing a blog post on fine tuning LLM, and I found this blog very helpful in understanding parameter-efficient LLM fine tuning with Low-Rank Adaptation (LoRA). The article explains how LoRA works and compares LoRA to other finetuning methods. In brief, LoRA is more efficient than full-fledged finetuning while achieving similar performance. 

[IV] 🌩LangGraph Cloud

LangChain recently released a new product called LangGraph Cloud. LangGraph, as readers of this newsletter will know, is a framework for building agentic applications. They claim LangGraph Cloud is the easiest way to deploy LangGraoh agents and it adds a bunch of functionality that is important when going to production with agentic applications. Some of the key features of LangGraph Cloud include assistants and threads, background runs, cron jobs, double texting support, and the LangGraph Studio. I particularly liked the LangGraph Studio, it is an IDE where you can debug, share and test out your LangGraph agents. 

[V] 👷LangGraph Engineer

As at the time of writing this sentence, they also introduced in their LangGraph Cloud a new feature called LangGraph Engineer (code), an AI agentic system that orchestrates your LangGraph graph for you. I think this is a great idea! It will make building these graphs more fun.

“This is an alpha version of an agent that can help bootstrap LangGraph applications. It will focus on creating the correct nodes and edges, but will not attempt to write the logic to fill in the nodes and edges - rather will leave that for you. 

LangGraph: The agent consists of a few steps: 1. Converse with the user to gather all requirements 2. Write a draft 3. Run programmatic checks against the generated draft (right now just checking that the response has the right format). If it fails, then go back to step 2. If it passes, then continue to step 4. 4. Run an LLM critique against the generated draft. If it fails, go back to step 2. If it passes, then continue to the end.”

[VI] ⌨Prompt Engineering is Dead

In this talk, one of the folks from Databricks discusses prompt engineering for LLM applications. The speaker discusses the limitations of traditional prompt engineering for LLM apps and advocates for the DSPy framework as a more effective alternative. Developed by Omar Khattab at Stanford, DSPy transforms the approach to working with LLMs by focusing on system design rather than manual prompt crafting. The framework separates core logic from textual representation, creating reproducible and LLM-agnostic modules. Key features of DSPy include the use of "modules" and "signatures" to define desired LLM behavior, an optimizer that automatically generates optimal prompts, and a systematic approach to building LLM applications in general. The argument is, by shifting focus from manual prompt engineering to robust system architecture, DSPy allows developers to concentrate on overarching system design and logic. 

I have heard a lot about DSPy but I haven’t just taken the time to read about the framework, initially it appeared to me to be orthogonal to what folks at LangChain are doing. But it appears, not quite!

[AI X Industry + Products] 🤖🖥👨🏿‍💻

[I]:  📹Google Vids

Google Vids is an online video creation app that is part of the Google Workspace suite. It was announced back in April, and is designed to help users create informational videos for work-related purposes. I recently got access to its experimental early version, I have tried it and at least for now its not working quite well. But I do think that it is a powerful product that will save a bunch of time once it starts working optimally. The promise here is that, at best, you can go from docs to vids, to put it in Google’s lingua. And to be clear, you can then go in and edit as you like, to match your taste.

[II]: 🚐GPT-4o Mini: Cheap A%$

OpenAI has released GPT-4o mini, a language model that's surprisingly powerful and affordable (as in, cheap A%$). It outperforms its predecessor, GPT-3.5 turbo, on many tasks, yet it's 60% cheaper. This video showcases its impressive performance -  its capabilities with examples of generating emails, extracting information from receipts, and processing Hacker News posts. Overall, GPT-4o mini is a promising tool that could change how we interact with AI, especially for those seeking cost-effective solutions.

[III]:  🦙 Llama 3.1: Biggest, Best Open-Source AI Model Yet 

“Llama 3.1 405B is the first openly available model that rivals the top AI models when it comes to state-of-the-art capabilities in general knowledge, steerability, math, tool use, and multilingual translation… As part of this latest release, we’re introducing upgraded versions of the 8B and 70B models. These are multilingual and have a significantly longer context length of 128K, state-of-the-art tool use, and overall stronger reasoning capabilities…”

Here, Zuck discusses the benefits of open-source AI and why Meta is releasing Llama 3.1, as open source. He argues that open-source AI will lead to a more advanced and secure ecosystem. He also believes it will be safer because it is more transparent. They hope that by releasing Llama 3.1 open source, it will become the industry standard. I think I need to be taking Llama more seriously. I haven’t built anything with it yet. That is about to change.

[IV]: 💻 Phind 

This podcast introduces Phind, an AI assistant that aim to help developers go from an idea to a working product. In essence, it is a search engine that answers questions with in-house fine tuned LLMs, targeted towards developers. The founder of the company talks about his journey building the product. You will like it more if you fancy AI engineering. I used Phind when it first came out a while back, however, I have pivoted to perplexity mostly now for my AI search. I have also tried Exa and Globe Explorer.

[V]: 👨‍🏫 LlamaTutor

An AI tutor built with Llama 3.1. In essence, it works by “entering a topic you want to learn about along with the education level you want to be taught at and generate a personalized tutor tailored to you!”

[AI + Commentary] 📝🤖📰

[I]: 👝There will be more AI agents than people

“Mark Zuckerberg says in the future there will be more AI agents than people as businesses, creators and individuals create AI agents that reflect their values and interact with the world on their behalf.”

https://x.com/tsarnick/status/1815862874272849942

[II]: 🕴 Getting AI to do the Work of 700 people

This conversation with folks at Sequoia is about how a company used LLMs to improve customer service. The speaker, Sebastian Siemiatkowski, is the CEO of Klarna, a financial services company. He talks about how Klarna began using an LLM developed by OpenAI to help resolve customer disputes. Siemiatkowski says that while LLMs are not very creative because they are trained on a massive amount of data and tend to find average solutions, they can be useful for tasks that are repetitive and require processing a lot of information, such as resolving customer disputes. Klarna started using this tech to help its customer service agents resolve disputes. The LLM was able to analyze the data from a dispute and suggest a resolution, which helped the agents resolve disputes more quickly and efficiently. In fact, their LLM was able to do the work of seven hundred full time customer service representatives.

Siemiatkowski also talks about the future of AI and how it could be used to generate new products. For example, he says that AI could be used to create images of products that do not yet exist. These products could then be produced on demand.

[III]: 💻 AI is a Services Revolution

In this essay, Rex Woodbury discusses how LLMs are poised to transform the services industry, which has become dominant in the modern American economy. The author argues that while AI disruption is happening, it's occurring slowly, with many enterprises still in the experimentation phase rather than full deployment. The essay outlines a formula for AI startups: choosing a text-heavy services industry, using LLMs to automate workflows, and leveraging industry-specific data for improvement. Examples of AI applications in law, healthcare, education, investment banking, and insurance are provided. The author emphasizes that despite the current hype and early movers in the field, the AI revolution is still in its early stages, likening it to the "Irruption Phase" of technological cycles. The essay concludes by stating that the race to dominate AI-powered services is just beginning, with ample opportunities for innovation and disruption across various industries. I happen to agree with these insights too, afterall most of the toolings for the disruption are still being actively built.

[IV]: 💻 Generative AI: A New Consumer Operating System

This research article from folks at Ark Invest is  about generative AI and its potential to change the way we interact with computers. It discusses how AI agents will simplify complex tasks and transform online shopping. Consumers will likely rely on AI assistants for most purchases. This will lead to a decline in traditional advertising methods. The article also predicts an increase in voice-controlled interfaces. However, screen-based devices will likely remain important. AI wearables are expected to become commonplace by the end of this decade. 

“The history of computing is defined by improving the productivity of individuals and enterprises. The transition from command-line interfaces (CLIs) in the 1970s to graphical user interfaces (GUIs) in the 1980s enabled the abstraction of complex syntax with visual icons and windows. Then, the flattening of the computing learning curve accelerated the adoption of personal computers (PCs) through the 1990s to spawn the World Wide Web and the internet applications built on top of it. At the turn of the last century, touchscreen phones and mobile operating systems placed the power of computing in our palms. Now, generative artificial intelligence (AI) is accelerating the adoption of digital applications and creating the next epochal shift in human-computer interaction.”

[V]: 📖 On Meta LLM Open Source Projects 

Fun discussion here with Meta’s Joe Spisak on Llama 3.1 405B and the Democratization of Frontier Models.

[VI] 🎙 Podcast on AI and GenAI

(Additional) podcast episodes I listened to over the month.