Juan Pablo Rivera

LLMs can output tokens 200% faster using multi-token prediction. And it’s now been open sourced by Meta. Here’s how it works — Multi-token prediction extends standard language modeling -- it predicts several future tokens at once instead of just the next one. - First, the model uses a shared core to make a context representation from its past tokens. - Then, it predicts multiple future tokens in parallel using separate prediction layers. This approach is much more memory-efficient, so during training, it can handle large vocabulary sizes without overloading the GPU. And at inference, it can decode much faster. Why does it work so well? According to the paper, it’s because multi-token prediction is able to reduce the mismatch between training and inference. This makes the model better at handling long-term dependencies. Paper: https://lnkd.in/ge_quFaS HuggingFace models: https://lnkd.in/gUgzsbtZ #AI #LLMs

7

Very useful paper to read on #LLM in #healthtech, #digitalhealth, and clinical care. Thank you, Jan Beger for all your excellence and informative summaries of #AI in #healthcare papers!

10

Ilya Sutskever confirmed at NeurIPS '24 pre-training scaling law will not be the same. Internet data, the AI fossil fuel, is running out. It is not the end of the AI hype, the rapid rise of foundational AI has outpaced our ability to fully grasp their long-term societal impact. AI agents and applications are poised to transform every facet of our lives, and we will see it happening in coming few years. There will still be improvements in model architecture, and synthetic data creation, so models will improve, just not at the lightning speed they did till now. Looking forward, the single most important milestone in AI is creating systems capable of advancing themselves—AI researchers that can innovate and improve foundational models autonomously. We need to assess how far are we from this stage.

9

Unpopular opinion: Most biotech companies aren't ready for AI/ML, and it's not because of their data scientists. It's because of their bench scientists. Here's why: AI/ML requires large, consistent datasets. But in most biotechs, every scientist analyzes their assays slightly differently. They use different thresholds, different ways of normalizing data, different ways of identifying outliers — even different formulas for basic calculations. This inconsistency makes it impossible to compare results across experiments, let alone build AI models. "But we have SOPs!" you might say. Sure, for running the experiments. But for analyzing the data? Rarely. The hard truth is that without standardized analysis protocols, your data is useless for AI/ML. And implementing those protocols means taking away some of the autonomy bench scientists are used to. It's a tough pill to swallow, but it's necessary if biotechs want to harness the power of AI. At Sphinx Bio, we're building tools that make it easy to implement and enforce these protocols, without sacrificing the flexibility scientists need for exploratory analysis.

477

56 Comments

This one struck a nerve, but there was some misinterpretation -- I'm not suggesting dry lab teams blame their wet lab counterparts! Rather, science is noisy and difficult, so let's not add additional sources of variation at the analysis steps. Standardize what you can and your AI/ML initiatives will be much better off. This requires better tooling, yes, but it also requires education and teams to work together. If your tools don't bring together your teams, then you probably have the wrong tools. The most successful companies of the next decade will be ones which blend software and science natively.

8

1 Comment

I spoke today with computational bio expert Jefferson Parker, PhD (at NullSet Informatics Solutions) for our State of AI in Biotech report. He shared some interesting career advice for those working in AI: 9 out of 10 leaders in biotech want AI in their scientific development, but do not know WHY. 1 out of 10 DO know why. They probably have the best data, and that is a blessed place to be and develop in your career. Always ask why leaders want AI and much more will be revealed. And for leaders, understanding why you want AI (beyond the hype and promises) will lead to a team culture that prioritizes data and let's the best talent flourish. It's win-win.

24

3 Comments

Looks like I've been quite lucky today - Found a 3rd article which seemed really valuable and worth sharing How can we be sure the LLM isn't generating something incorrect or gibberish? Can't just give an unevaluated output to our user, now can we! Santiago Valdarrama has mentioned a few resources that can help set up an evaluation pipeline to assess the quality of the LLMs outputs. #goodread #datascience

2

Foundational models, or large language models (LLMs), train at the scale of the entire internet to acquire the vocabulary, syntax, and narrative structure of human language. Fine-tuning involves using a smaller, domain-specific dataset to grant additional specificity to a pre-existing foundational model that has already acquired patterns and features from massive underlying training datasets. By utilizing the existing knowledge embedded in a pre-trained language model, we unlock high-performance language understanding, but fine-tuning helps to get the right answer - not just an answer that sounds right - with a minimum of hallucinations or incorrect info. In healthcare, out-of-the-box, foundational models are often too generic to sufficiently answer the precision of a healthcare query. Foundational language models are trained on a massive scale of data, which includes jokes, incorrect information, misunderstandings, and misinformation, along with the sum total of human knowledge. The problem is, if you ask a large language model to give you factually correct information when it has been trained on BOTH accurate and inaccurate information - you’re playing a bit of a dangerous game. Enter fine-tuning language models. In the old days, we used to build a single model for a single task - for example, classifying procedures into evidence-based or not as laid out by clinical practice guidelines, for patients with particular disease markers. Each disease would have its own model, which took a long time to build and didn’t scale particularly well, and we struggled with the sparsity and inherent high-dimensionality of healthcare data. By fine-tuning LLMs, however, we're able to specialize those foundational models to achieve greater domain specificity and specialization, leverage the unique predictive and planning capabilities of LLMs, and cut down on the incorrect information that such models inevitably produce, while still preserving the underlying language prowess of the LLM. If improved accuracy and specificity wasn't enough of a selling point, there's another notable upside to fine-tuning foundational models for use in a clinical context. In terms of computational efficiency, using LLMs out of the box on specific tasks can require massive compute (and spend) that amounts to computational overkill for the problem. Some LLMs charge per query, some by compute load or time - either way, as a business running LLMs in a production environment, minimizing the number of LLM calls means optimizing your compute expenditures, too. When LLMs fail at scale in healthcare, it's usually along the fault line of failure to specify adequately. Fine-tuning large language models along domain-specific lines is one way to effectively productize genAI for healthcare use cases. #genAI #LLM #ai #healthcareAI #healthtech #largelanguagemodels #generativeAI #chatGPT

61

2 Comments

This paper examines the economics and equity challenges of implementing LLMs in healthcare, focusing on three pathways: training from scratch, fine-tuning pretrained models, and using out-of-the-box models, and evaluates their trade-offs in cost, customization, and accessibility. 1️⃣ Training from scratch offers maximum customization and transparency by building LLMs on specific healthcare data but requires significant resources in infrastructure, workforce, and upfront costs. 2️⃣ Fine-tuning pretrained models balances cost and customization by adapting existing LLMs with healthcare-specific data but risks biases from the original training data and has limited transparency. 3️⃣ Using out-of-the-box models enables rapid deployment with minimal infrastructure and workforce needs but offers limited customization and transparency, often relying on general-purpose LLMs. 4️⃣ Cloud service providers play a critical role in scaling LLMs through managed services, federated learning, and hardware accelerators, but affordability and equitable access remain challenges. 5️⃣ Privacy, ethics, and regulatory considerations, including data deidentification and compliance with standards such as HIPAA and GDPR, are essential for LLM adoption in healthcare. 6️⃣ Potential applications of LLMs include chatbots, clinical decision support, and data summarization, but ongoing efforts are needed to mitigate biases, toxicity, and ensure model transparency. ✍🏻 Radha Nagarajan, FHIMSS, FAMIA, Midori Kondo, Franz Salas, Emre Sezgin, Yuan Yao, Vanessa K., Sandip Godambe, Naqi Khan, MD, MS, Alfonso Limon, Ph.D., Graham Stephenson, Sharief Taraman, MD, DABPN, DABPM, FAAP, Nephi Walton, Louis Ehwerhemuepha, Jay Pandit, Deepti Pandita, Michael Weiss, Charles Golden, Adam Gold, John Henderson, Angela Shippy MD, Leo Anthony Celi, Bill Hogan, Eric Oermann, Terence Sanger, Steven Martel, MD. Economics and Equity of Large Language Models: Health Care Perspective. Journal of Medical Internet Research. 2024. DOI: 10.2196/64226 📄 Planning to submit your research to a peer-reviewed, open-access journal? Use the discount code AMJB200 at checkout to save $200 USD on the article processing charge at any JMIR Publications journal. Start your submission here: https://lnkd.in/deuAEN2r. It's important to me that you know this isn't a sponsored post. I'm simply committed to keeping you informed about opportunities to disseminate your research effectively.

111

16 Comments

🎯Data was, is, and will be the only fuel. Analogy with the fossil fuel is a good one, since the data we have today is as dirty as oil and coal. What big tech companies tried is to put a lot of "garbage" data, and what they did get was fantastic. Yet, we are at the point of diminishing returns. Data is the beginning and the end of this cycle.” Great thread, particularly this comment 👆from Ilija Lazarevic

20

2 Comments

🚀 Advancing Information Retrieval with APEER: Automating Prompt Engineering for LLMs! 🚀 I recently delved into a fascinating paper on APEER (Automatic Prompt Engineering Enhances LLM Reranking), a method transforming the way we approach relevance ranking in Information Retrieval (IR) using Large Language Models (LLMs). 🌟 🔍 Challenges in IR with LLMs: Current IR methods with LLMs often rely heavily on human-crafted prompts for zero-shot relevance ranking. This process is time-consuming, subjective, and lacks scalability. The complexities of integrating query and passage pairs and conducting comprehensive relevance assessments further complicate the effectiveness of existing methods. 💡 APEER: A Game-Changer APEER addresses these challenges by automating prompt engineering through iterative feedback and preference optimization. Here's how it stands out: Feedback Optimization: Continuously refine prompts based on performance metrics. Preference Optimization: Improves prompts by learning from positive and negative examples. 🌟 Key Highlights: Reduced Human Effort: APEER minimizes the need for manual prompt crafting, making the process more efficient and less reliant on human expertise. Enhanced Performance: Demonstrates significant improvements in LLM performance on IR tasks, with notable gains in metrics like nDCG@10. For example, APEER achieved an average improvement of 5.29 nDCG@10 on eight BEIR datasets over manual prompts on the LLaMA3 model. Better Transferability: Shows consistent outperformance across diverse datasets and LLM architectures, including GPT-4, LLaMA3, and Qwen2. 📊 Robust Validation: APEER has been rigorously tested on multiple datasets such as MS MARCO, TREC-DL, and BEIR, ensuring its robustness and effectiveness in various IR scenarios. 🌐 Implications: This advancement represents a significant step forward in optimizing LLM prompts for complex relevance ranking tasks. By reducing manual intervention and enhancing the efficiency of LLMs, APEER paves the way for more scalable and accurate applications in real-world IR tasks. 🛠️ Conclusion: APEER's automated approach to prompt engineering is a major leap towards more effective and scalable IR systems. It highlights the potential of integrating iterative feedback and preference optimization to refine LLM performance, offering promising avenues for future research and practical applications. Looking forward to seeing how APEER influences the landscape of Information Retrieval and beyond! #InformationRetrieval #LLM #AI #MachineLearning #NLP #DataScience #Research

1

Continuing on learning more about how to prompt engineering in a more systematic way, I read “Large Language Models as Optimizers” by Yang et al., 2024 which was recently recommended by a friend when talking about textgrad (check it out in my previous posts). In this paper, the authors propose an approach to treat LLMs as optimizers for a task to optimize that is described in natural language. In each optimization step, the LLM generates a new solution using the prompt that contains previous solutions and evaluated. In the next iteration, the previous solution is also integrated in the prompt, and so on. The authors argue that this approach outperforms human generated prompts for certain tasks. This idea reminds me of textgrad and DSPy where the prompt is modified to maximize task performance by systematically changing the prompt (kind of following a text gradient). To me, the interesting part of this idea is how to change the prompts and how to define the evaluations for each task so that this approach works. Paper: https://lnkd.in/gFrcPaxZ Enjoy!

14

Snowflake’s BUILD 2024 AI/ML Development Conference Highlight – AI, Agents, and Data by Andrew Ng (Founder of LandingAI) The session centered on the evolution of AI with a specific focus on AI agents and the emerging paradigm of agentic reasoning. This new approach emphasizes the ability of AI to engage in iterative, goal-directed workflows, fundamentally transforming how applications are developed and deployed. With the exponential growth in the volume and complexity of data, the role of unstructured data—text, images, audio, and video—is becoming a cornerstone in driving automation and innovation across industries. Ng's discussion explored how this convergence of AI agents, agentic reasoning, and unstructured data is poised to revolutionize automation and application development in a cost-effective and scalable way. Recent advancements in large language models (LLMs) and large multimodal models (LMMs) have drawn significant attention due to their broad applicability and power. Building on these foundational technologies, AI agents are emerging as the next significant breakthrough. These agents, designed to reason and interact iteratively, are proving transformative in industries that require complex decision-making and operational efficiencies. As the emphasis shifts from models to applications, the importance of effective data engineering and the integration of unstructured data in workflows will shape the next generation of AI-driven solutions. Video Link: https://lnkd.in/e2njUdFC #ArtificialIntelligence #AIAgents #AgenticReasoning #LLMs #LargeLanguageModels #Innovation #LargeMultimodalModels #LMMs

9

Grateful to William Sun and MIT Club of Northern California for organizing an insightful Gene Therapy Conference. Being new to this field, here are some things I learned: 🔬 Delivery of gene therapy treatments is a big challenge – ensuring that they reach the right cells, at the right dose, without causing harm, remains a big hurdle. 💡 The up-front cost of gene therapy can be high (e.g., $3M+), however when weighed against a lifetime of treatments using other expensive drugs it can actually be more economical. ⏳ After receiving a gene therapy treatment, it’s difficult to undergo another. Patients face the tough choice of opting for a moderately effective treatment now or holding off for a potentially better one in the future. 🔍 Advances in DNA testing and treatments could lead to removing and repairing stem cells before a disease even presents itself. 🎯 The sweet spot for gene therapy products lies in targeting niche diseases that also offer broader health benefits to larger populations. 🤝 More transparency around FDA submissions, protocols, and research would streamline progress across the entire ecosystem and be a win for all players + humanity. Special thanks to the panelists for their stories and insights, specifically: Trevor Martin (CEO, Mammoth Biosciences), Amy Pooler (VP and Head of Research, Sangamo Therapeutics, Inc.), and Agnieszka Czechowicz, MD, PhD (Physician Scientist, Stanford University School of Medicine). 🙏🙌

7

1 Comment