The AI Agent Index investigates 30 top AI bots and finds just four have published formal safety and evaluation documents relating to the actual agents. The new wave of AI web browser agents, many designed to mimic human browsing, have the highest rates of missing safety information.  

 

A new method can test whether a large language model contains hidden biases, personalities, moods, or other abstract concepts. MIT researchers can zero in on connections within a model that encode for a concept of interest, to improve LLM safety and performance.

Qiang Li and colleagues have created “cyborg” pancreatic organoids that combine stretchy miniature electronics with stem cell-derived pancreatic islets, using the implanted electronics to monitor electrical activity related to glucose regulation in maturing α and β cells. Islet α and β cells secrete glucagon and insulin hormones, in response to electrical changes in the cell membrane. The researchers also used the electronics to stimulate the cells to enhance their glucose responsiveness and show how that responsiveness changes as the cells mature, how it is affected by different chemical compounds and circadian hormones, and how it is related to gene expression. Together, the cyborg organoids offer a path to study the biology and trace the maturation of α and β cells at the single-cell level, with an eye toward potentially treating or engineering new pancreatic islets in people with diabetes. In a related Perspective, Jochen Lang and Matthieu Raoux discuss how the cyborgs could be used to guide the growth of mature human pancreatic organoids for regenerative medicine and other applications.

AI models have their own internal representations of knowledge or concepts that are often difficult to discern, even as they are critical to the models’ output. For instance, knowing more about a model’s representation of a concept would help explain why an AI model might “hallucinate” information, or why certain prompts can trick it into responses that dodge its built-in safeguards. Daniel Beaglehole and colleagues now introduce a robust method to extract these representations of concepts, which works across several large-scale language, reasoning, and vision AI models. Their technique uses a feature extraction algorithm called the Recursive Feature Machine. By extracting concept representations with this technique, Beaglehole et al. were able to monitor these models in ways that exposed some of their vulnerabilities to behaviors like hallucinations and to steer them toward improved output responses. Surprisingly, the concept representations were transferable between different languages and could be combined with other concept representations for multi-concept steering, the researchers noted. “Together, these results suggest that the models know more than they express in responses and that understanding internal representations could lead to fundamental performance and safety improvements,” the authors write.