ε Pulse: Issue #19
Replit Agent 🕵️, Act One 📹, and The Machine of Loving Grace 🤖.
My Other Publications:
Around the Web Issue #32: The Death of Immortality Dreams.
Podcast
Note: this was AI generated with NotebookLM, read the actual newsletter and source materials, as applicable, for grounding.
In this newsletter:
🧬
AI Nobel Prize Round Up
Biomedical Discovery with AI agents.
Remote Controlled Life Sciences Lab
👨💻
Multi Agent Architectures
Engineering Replit Agent.
Magentic-One: Generalist Multi-Agent System
🏭
How to Use OpenAI o1
Runway ML Act One
Midjourney Web Editor
Eleven Labs Conversational AI Agent
📝
Export Your Brain with AI.
Dario Amodei: Machines of Loving Grace
The AI Future Has Arrived
[LLM/AI for Science et al] 🤖 🦠 🧬
[I]: ⚕ AI Nobel Prize Round Up
The 2024 Nobel Prizes in Chemistry and Physics emphasize the expanding role of AI in advancing scientific research. In Chemistry, David Baker received half of the award for his pioneering efforts in computational protein design. The other half was awarded jointly to Demis Hassabis and John Jumper for their work on AlphaFold, an AI system that has transformed the field of protein structure prediction. Baker’s innovations have led to the design of novel proteins with potential applications in medicine and materials science. Meanwhile, AlphaFold has provided solutions to a decades-long problem in biology by predicting protein structures with exceptional precision.
The Nobel Prize in Physics honored key contributors to the development of machine learning. Geoffrey Hinton and John Hopfield were recognized for their foundational research in artificial neural networks. Hinton’s contributions to backpropagation and deep learning have been fundamental to the rapid growth of AI, enabling breakthroughs in areas such as image recognition and natural language processing. Hopfield’s earlier work on associative memory networks laid essential groundwork for the development of modern AI technologies. These awards highlight the profound impact of AI across multiple scientific fields, signaling a new era where computational and AI-driven approaches play a central role in scientific innovation.
I also enjoyed listening to this podcast from a16z about the historical trajectories of some of the discoveries highlighted above. In addition to synergy between computational sciences and fundamental sciences.
[II]: ⚕ Biomedical Discovery with AI agents
I really enjoyed reading this (very long) paper on biomedical discovery with AI agents. There are many things going on in the review but I would just highlight two themes in the paper. The work delves into the essential components and varying degrees of autonomy required for AI agents to exhibit agentic behavior—where agents can act independently and purposefully.
The research outlines a spectrum of autonomy, from reactive systems with minimal decision-making capabilities to fully autonomous agents capable of complex reasoning and long-term goal pursuit. Key to this autonomy is the seamless integration of four core modules: perception, interaction, memory, and reasoning. Perception allows agents to interpret and make sense of their environment, while interaction facilitates meaningful exchanges with other agents or systems. Memory serves as the foundation for retaining past experiences and information, enabling learning and adaptation over time. Finally, reasoning enables the agent to process information, make decisions, and plan actions that align with its goals. All of these were discussed in the context of biomedical sciences.
The paper emphasizes that achieving the various levels of agentic AI requires a balance between these modules, tailored to the specific level of autonomy required by the task. For example, low-level agents with basic perception and interaction capabilities might excel in simple, task-specific environments. In contrast, higher-level agents—those capable of strategic reasoning—depend heavily on memory for contextual understanding and on interaction to refine decision-making through feedback.
[III]: ⚕ Remote Controlled Life Sciences Lab
I was chatting with a friend the other day and he mentioned the Emerald Cloud Lab. It was the first time hearing of the company, you can watch a demo on their website. The ECL platform provides researchers with the capability to design and execute experiments remotely via their ECL Command Center, a software interface. By sending their samples to an ECL facility outfitted with specialized instruments, scientists benefit from a lab that operates continuously, greatly boosting experimental throughput beyond what traditional labs offer. This remote access allows experiments to be conducted from any location with internet connectivity, and the platform’s automation enhances efficiency, enabling scientists to carry out several times more experiments compared to conventional laboratory settings. Additionally, the platform’s precise automation and detailed digital records ensure high reproducibility, making it easy to replicate experiments accurately.
[AI/LLM Engineering] 🤖🖥⚙
[I]: 🕵️ Multi Agent Architectures
In this video, Harrison Chase talks about the different architectures for building LLM-based agentic AI systems. The video starts by defining what a single agent system is and then discusses some of the common issues with single agent systems, such as having too many tools, context growing too complex, and the need for multiple specialization areas. The video then discusses the benefits of multi-agent systems, such as being more modular, more specialized, and having more control.
Harrison Chase then covers some of the common architectures for multi-agent systems, including single agent systems, network of agents, supervisor agents, supervisor with tools agents, hierarchical agents, and custom cognitive architectures. Chase also discusses how agents communicate with each other, including sharing an overall state object and passing the results of a tool call.
He also gives a brief overview of how to have two agents with different states communicate with each other and how to have two agents communicate on the same list of messages.
[II]: 👨💻Engineering Replit Agent
I haven’t used Replit Agent, but I have watched a few demos. Replit Agent is an “AI-powered development tool that acts as a virtual pair programmer, allowing users to create, debug, and deploy full-stack applications from natural language prompts, handling tasks like configuring environments, installing dependencies, and executing code.”
In this article, they explained how the agent was built, and some of its bespoke functionalities. In brief, the agent supports multi-step task execution, manages infrastructure, and eases the build-experiment-test-deploy process. It uses a multi-agent architecture with a manager agent, editor agents, and a verifier agent to ensure reliability and user involvement. The Replit team employed several techniques to enhance the performance of their coding agents, such as few-shot examples, long task-specific instructions, tool calling, structured formatting, and dynamic prompt construction.
Here is the CEO of replit talking about the product on YC podcast.
[III]: 🔖 Magentic-One: Generalist Multi-Agent System
In the spring of this year, I spent some time playing with crew AI and I was quite shocked as to how unstable it could be, and it is expected, it is a high level agentic AI framework. My reaction to crew AI performance led me to embrace LangGraph, a low level orchestration framework for building agentic AI systems, and that explains the preponderance of Langraph content in this section of the newsletter.
Lately Microsoft released a generalist multi-agent system called Magentic-One, which appears to be a middle ground between frameworks like crew AI and LangGraph, because let’s face it, there is a great deal of learning to be done in mastering LangGraph.
This video does a good job describing Magentic-One. It consists of an orchestrator agent and a number of sub-agents. The orchestrator agent acts as the lead agent, deciding which sub-agent to use next. The sub-agents are responsible for carrying out specific tasks, such as browsing the web, writing code, or interacting with the computer terminal. To use Magentic-One, the user simply needs to provide a high-level description of the task they want to accomplish. The orchestrator agent then breaks down the task into smaller steps and assigns each step to a sub-agent. The sub-agents then carry out their tasks and report back to the orchestrator agent. The orchestrator agent tracks the progress of the task and makes adjustments as needed.
[AI X Industry + Products] 🤖🖥👨🏿💻
[I]: 🧠 How to Use OpenAI o1
I was part of the crowd that switched from chatGPT to claude when Anthropic released claude sonnet 3.5 and Artifacts, I am also among the folks who increasingly have moved in the opposite direction due to such features as Canvas and models like o1 preview. But what I quickly realize is that it is not obvious how to use this model. I have found many resources useful in learning about what is capable of. Here are a few:
OpenAI is crazy good - here’s how you should be using it.
Estimating the cost of discoveries: Andrew White here estimates the cost of scientific discoveries over time by using Wikipedia articles and OpenAI's 01-preview model. The 01-preview model is used to estimate the cost of a discovery by taking into account the number of man-hours involved, the capital expenditures, and the time period in which the discovery was made. A very interesting analysis indeed.
[II]: 📹 Runway Act One
I have explored using runway ML video generation products for some of my video essays in the past, but I ended up discontinuing using it because the ‘return on investment’ is not that great. But one of their latest features caught my eye: Runway Act-One [Link].
“Act-One can create compelling animations using video and voice performances as inputs. It represents a significant step forward in using generative models for expressive live action and animated content.”
So many fun opportunities here.
[III]: 📷Midjourney Web Editor.
My go-to text to image is Midjourney, and I was quite excited with the release of the image editor tool.
“The image editor lets you upload images from your computer and then expand, crop, repaint, add, or modify things in the scene. [There is also the] ‘image retexturing mode’ that estimates the shape of the scene and then ‘retextures it’ so that all of the lighting, materials, and surfaces are different. All image editing actions can be controlled via text prompting and region selection. The editor is compatible with model personalization, style references, character references, and image prompting.”
See more.
[IV]: 🗣️Eleven Labs Conversational AI Agent.
ElevenLabs has introduced a beta platform for creating customizable conversational AI agents that combine speech-to-text, language models, and text-to-speech technology. Users can design agents for a range of roles, including customer service, educational tutoring, retail assistance, and interactive storytelling. The platform offers key features like model selection from major providers, voice customization with cloning options, multilingual support, and knowledge base integration. It also includes deployment options for website integration and tools to monitor agent performance, aiming to make AI interactions more natural and versatile for various digital applications.
Here is a video work-through.
[AI + Commentary] 📝🤖📰
[I]: 🧠Export Your Brain with AI.
Interesting take on AI companions and the AI Brain vertical niche:
“There are clearly real use cases for an “AI brain.” But in my experience, ChatGPT isn’t the ideal product. It’s meant to be a helpful, general assistant – which it’s great at – but it’s not optimized to be a personal or professional companion. A few places where it falls short: memory is fairly rudimentary, it can’t “view” your screen, it doesn’t proactively reach out to you, and it’s limited in modality (text and voice).
My early take is that this AI companion should be able to ingest information about you across all content modalities: text, image, and audio. Some people will want to text with it, others will want to call, and most will end up simply sending screenshots for analysis. TBD how this happens – is it just an app? A hardware product, like glasses or a pendant?
Especially in work contexts, it’s important that these products can “see” what you’re doing. It might sound invasive today, but so did capabilities like location sharing or facial recognition in your photos app when they were first introduced. I think we’re not far from a reality where most people have an always-on AI that’s viewing their screen. Your daily tasks, emails, or even Slack messages provide valuable context.”
[II]: ⚙ Machines of Loving Grace
This essay - by Dario Amodei, CEO of Anthropic - is a long and well-written essay on a possible AI world in our future. It is not solely about science, but because science is so fundamental to the progress of humanity and the human person, the essay deals with science in a proportionate manner. I have only read the introduction and the part on biology and health, so I cannot speak to the other parts of the essay.
Here is one relevant part of essay:
“… my basic prediction is that AI-enabled biology and medicine will allow us to compress the progress that human biologists would have achieved over the next 50-100 years into 5-10 years. I’ll refer to this as the “compressed 21st century”: the idea that after powerful AI is developed, we will in a few years make all the progress in biology and medicine that we would have made in the whole 21st century.”
[III]: 🤖 The AI Future Has Arrived
Any frequent Epsilon reader will not find this Y combinator podcast episode particularly insightful, which is not to say it is not insightful, because it is. The point being: a frequent reader of the blog will know at this point that AI is terribly important.
In brief: AI is no longer a fad. It's here to stay, and it's changing everything. Caldwell and Seibel from YC discuss what you should be doing to take advantage of this opportunity. If you're looking to start a company, this is the perfect time. You should also choose your job wisely and make sure you're working for a company that's embracing AI. In addition, you need to learn how AI works and start using AI tools in your everyday life. Finally, don't just sit there and do something. Take action and get involved. The future is here, and it's powered by AI.
[IV] 🎙 Podcast on AI and GenAI
(Additional) podcast episodes I listened to over the past few weeks