Narayana P.

LLM Data Engineering Pattern 3: Aggregating Unstructured Insights at Scale Modern financial analysis often requires synthesizing information from a vast array of unstructured sources—regulatory filings (e.g., 10-K, 10-Q), research reports, market commentary, and even audio transcripts of earnings calls. In many cases, these sources must be combined to form a comprehensive view of an industry’s risk profile. Historically, analysts have relied on manual selection and review of documents, a labor-intensive process that does not scale well in today’s data-rich environment. Consider analyzing credit risk in the U.S. commercial real estate sector: 1) Regulatory Filings: An LLM can systematically parse through hundreds of 10-K and 10-Q filings to extract commentary on asset valuations, loan-to-value ratios, or refinancing challenges. 2) Research Reports: It can synthesize qualitative assessments from research houses covering property market trends, default probabilities, or vacancy rates. 3) Earnings Call Transcripts: It can highlight executives’ forward-looking statements regarding rental income stability, interest rate exposures, or occupancy forecasts. By integrating these unstructured data points, an analyst can address questions like: “Given the commentary on sector vulnerabilities in these filings, research reports, and macroeconomic indicators, what are the key risk factors influencing the commercial real estate market?” Financial people, how are you doing this today? #llm #dataengineering #llmdataengineering #financialdata #finance

20

1 Comment

AI memory is fundamental for conversational AI and AI agents. It is critical for building personalized experiences and improving reasoning. Check out Letta’s course to understand how to use AI memory to enhance the customization and quality of your AI app.

15

Interesting answer from AlphaSense CEO Jaakko (Jack) Kokko on why they acquired Tegus - all about access to proprietary data, in this case the transcript library on private companies. You recently acquired a company called Tegus. Was this about having more proprietary data to feed into your models? First and foremost, it was about providing our customers with the best possible intelligence on the market, especially on private companies. A couple of years ago, we acquired a company called Stream, the No. 2 player in what you might think of as a library of interview transcripts between investors and experts on a company — their former executives, competitors, customers, partners. It allowed us to produce information that you can’t access anywhere else, and our customers loved it. Tegus is the market leader in that sector and was the natural next step for us. They have the world’s best library of private company content, and we immediately wrapped our search capabilities, our generative AI, our monitoring capabilities around their content and made it much more valuable. We now have the broadest coverage of insights on both public and private companies, allowing our AI to provide even better answers to our customers.

161

8 Comments

Last fall Ibbaka research found that the most common pricing metric for AI applications wasper user. This was confirmed in a recent survey by Kyle Poyar and Palle Broe for Growth Unhinged. What are the reasons for this? https://lnkd.in/gNzyy3r6 #pricing #pricingmodel #pricingmetric #AI #generativeAI

6

4 Comments

rumor I’m hearing: good chance there will be no more step function improvements in base ai models source is a founder building agents on top of base models; he also has connections at base model co's a few takeaways from our convo: 1/ base model intelligence isn't mature enough for pure autonomous agents to work well 2/ a significant step function improvement in base model intelligence would be required for them to work well ... but could be as far out as 12-18 mo away (if they come at all) 3/ autonomous agents were working fine for smaller and easier workflows, but if tasks are complex agents don’t solve reliably. this is leading to lots of customer churn. personally, i think even if the models don't improve much, still think there's tons of innovation and great companies to build what do you think? -- follow me Mike Hinckley

15

PE - How to Use AI to Unlock Growth in Portfolio Companies All PE firms and portfolio companies should be thinking about unlocking GTM and growth via AI (i.e. not merely ChatGPT or other LLMs but actually implementing apps that are "AI Agents"). 1. What are "AI Agents"? - AI agents are autonomous systems powered by ML and natural language processing, designed to perform tasks with minimal human intervention - They can interpret data, determine the workflow & direction (i.e. make decisions), and execute actions across various functions, continuously improving through self-learning 2. Examples of using AI Agents in PE portfolio companies: - For Sales: AI agents handle lead qualification, respond to inquiries, book meetings, and provide personalized product recommendations, ensuring no opportunities are missed - For Marketing: Generate campaign briefs, segment audiences, create tailored content, and analyze performance to optimize outreach efforts - For Customer Success & Support: Provide 24/7 support, resolve customer issues autonomously, escalate complex cases to human agents with full context, and ensure consistent, brand-aligned responses 3. How PE portfolio companies benefit: - Efficiency: Automate repetitive tasks, enabling teams to focus on high-value activities - Scalability: Handle growing customer volumes without increasing headcount - Personalization: Deliver tailored experiences using real-time data insights - $ Cost Savings: Reduce operational costs by streamlining workflows 4. Business Impact for PE Portfolio Companies: - Accelerate GTM Execution: Faster lead conversion, better customer retention, and enhanced marketing ROI - Improve Metrics: Boost Customer NPS, CSAT, and efficiency - Enable Growth: Scale operations seamlessly while improving team productivity. 5. How PE Firms Must Think About Adoption - Define clear portfolio objectives to use AI agents (not just LLMs) - Ensure integration with existing CRM and data systems - Prioritize high-quality data and ongoing performance monitoring - Use AI agents to complement existing GTM teams, focusing them on automation ------------------ #pe #privateequity #investing #ai

14

Thanks for sharing these insights. What I am seeing is that orgs test an idea using a proprietary #LLM like #OpenAI, then put it into production with an open source LLM using a development platform like #Llamaindex. That way, private data remains private.

1

LLMs are trained on internet-scale data. But to make AI work for work, models need to be grounded in your enterprise data -- a much harder problem than meets the eye! How do you connect to all your data sources and ingest them frequently enough while managing compute/storage costs? How do you make sure that you only show users results from documents they have access to? How do you rank results without internet-scale click and link data? How do you process the content of documents that are often hidden in PDFs and presentations? These are tricky problems, and it's why enterprise search didn't work for nearly a decade, until Glean came along. Turns out it's the exact same set of challenges needed to give LLMs the right context to work their magic on your enterprise content. Glean spent 5 years building the ingestion, search, and ranking infra needed to solve this problem well. This has given them the right to run the "Work AI" race and delight users at work. CEO Arvind Jain shares more on the latest episode of Training Data. Listen here, or wherever you get your podcasts: YouTube: https://seq.vc/ytd Apple: https://seq.vc/pit Spotify: https://seq.vc/jnz 

203

10 Comments

I've adapted Exa's semantic search API for Google Apps Script since I found myself frequently combining both tools. ▪️ Matches official SDK interface ▪️ Works natively in Google Workspace ▪️ Supports all API features e.g. neural search, content categorization, domain filtering, and text summarization, publish date. ▪️ Ready to use in Sheets, Docs, and Forms (Google Apps Script is not the first choice when we think about building integrations but after working with it on several projects, I found it quite powerful for specific use cases - simple, almost free, easy) Link to Github documenation and examples: https://lnkd.in/dXeAHhBy Tweet with 8 usecases e.g. Google Calendar brief generator, Google Form response enrichment (when someone submits a company name or topic in a form, automatically enriches the response with latest news, social media presence, and key information): https://lnkd.in/d_KSHgHN  

11

1 Comment

Incisive analysis by Sequoia partner David Cahn on the missing $600 billion in AI revenue... Basically, given what companies are spending on Nvidia's chips and infrastructure build out, a certain amount of resulting revenue is implied. There is a $600 Billion gap. This week's The Economist arrived at a similar conclusion. The ROI will likely be realized after 2032! Links below. #ai #chatgpt #nvidia #venturecapital #startups

8

1 Comment

Slowly slowly, then all at once...

13

Sure, the opportunity to build AI Agents is there, but existing SaaS companies have a major advantage: their distribution channel. It’s easier to experiment providing agentic solutions (under their tailored business model) if you have an existing customer base and deep domain expertise, otherwise brace for 360° disruptive competition.

10

1 Comment

How should OpenAI price Strawberry or o1 as it is now known. Flash poll results. Total of 74 responses from Friday Sept. 13. Interested in how generative AI applications are being monetized? Join Kyle Poyar from OpenView and Growth Unhinged, Michael Mansard from Zuora and Steven Forth from Ibbaka for an engaging roundtable on Sept. 19. Please register here. https://lnkd.in/gZNZqS6u My own thoughts is that this depends on strategic intent. Strawberry is a very different application than vanilla ChatGPT and it will provide value in new ways. It should not be tied to ChatGPT pricing. I would position it as a separate offer. However, part of the intent of Strawberry seems to be to provide training data for future reasoning models. In that case the more people that use o1 the better and pricing should be neutral. #Strawberry #Openai #generativeAI #genAI #pricing #monetization #o1

7

3 Comments

Is the AI gold rush is cooling off? 📉❄️ Remember when VCs were rushing to invest billions in foundation models? Some reporters are calling that we are close to the top of the market, and maybe heading down. 💸🚫 Eric Newcomer's latest piece reveals a shift in investor sentiment: "Venture capitalists no longer want to dump money into companies that are building their own foundation models... It's now clear that Meta is planning to spend billions to keep its open source model Llama competitive. OpenAI, Anthropic, Microsoft, and a few others are relevant here — but the foundation model business is too expensive for even well-funded startups." What we're seeing with our AI startup clients: +AI startups are more expensive to run. We were quoted in the NYT saying they spend 2x on compute/hosting vs. traditional SaaS. 💻💰 +Despite the cooldown, AI startups (even those building models) are still flush with cash. 💼💵 +Not all AI is LLMs. We see promising work in vision, drug discovery, and more. These areas remain costly but potentially groundbreaking. 🔬🧪 What's next for AI startups? The game is changing, but opportunities remain. 🤔💡 Check out the 1st comment for a link to the Newcomer article, and consider subscribing - it's a great newsletter.

12

1 Comment

🚀 Nuclia delivers the first-ever open-source RAG evaluation model—a crucial tool for the entire RAG community! 🚀 Retrieval Augmented Generation (RAG) methods have revolutionized document search, leveraging the latest advancements in Large Language Models (LLMs) and Deep Learning. However, evaluating these complex systems can be challenging due to the intricate nature of LLM outputs. At Nuclia, we’ve addressed this challenge with REMi, our efficient open-source LLM fine-tuned specifically for RAG evaluation. Alongside REMi, we’ve also developed nuclia-eval, an open-source library designed to simplify the assessment of RAG pipelines using REMi. https://lnkd.in/dBPHgayA

4

Monetizing generative AI functionality will be a key challenge in 2025. Ibbaka's second AI Monetization Survey digs deep into how companies are approaching this, probing on market strategy, growth motion, investment and pricing. Please take the survey and contribute your insights. The 2024 survey was taken by 300 companies and has been downloaded more than 1,000 times. https://lnkd.in/gWyGzAet #AI #pricing #monetization

1

AI pricing is hard. Customers (esp enterprise) need to do annual budgeting. They won’t sign up for “who knows what it will cost???” But the AI provider (whether OpenAI or something built on it) have highly variable costs based on usage. In addition, if AI increases productivity, it may **reduce the number of user accounts a company buys**. As you provide more value, you (as the AI provider) can see less revenue. No one has entirely solved this.

4

3 Comments

Large LLM API providers are rapidly lowering the cost of using their LLMs. In the absence of a clear leader in model quality, price has become the main competitive factor. Also most likely they see much larger revenue potential from B2B market and integration of LLMs. 🛠️ On August 12th, Google will drop the prices of Gemini Flash by 80%. This seems to be a response to the GPT-4o mini, just two weeks after its release. It would be great if they did the same with Pro 1.5. 🛠️ OpenAI The new version of GPT-4o has seen its price drop to $2.5 input/$10 output per million tokens, compared to the previous versions, which were $5 input/$15 output per million tokens. Over the past month, Claude 3.5 Sonnet and Llama 3.1 405B have significantly pressured OpenAI on pricing, forcing a response. 🛠️ DeepSeek is releasing the Context Caching they promised a month ago. The prices are astonishing: the cost of tokens when hitting the cache drops not by 2x, like Google’s, but by 10x, with free storage included. Also Google recently announced not only a price drop for Gemini Flash but also made model tuning available to all developers. I missed this announcement earlier, but it’s a game-changer: - Tuning is entirely free—not just for Flash, but also for Pro. - The inference cost for tuned models is the same as for the standard models (unlike OpenAI, where tuned models cost twice as much). This likely works through a method similar to batched LoRa, which means the quality of tuning on large datasets might be affected. However, this needs to be tested to understand the impact. Additionally, the Gemini API can be used for free within certain limits, including the ability to tune models based on your own requests. Here are some relevant hashtags: #AI #LLM #MachineLearning #APIs #GoogleGemini #OpenAI #GPT4 #DeepLearning #TechNews #AITrends #AIInnovation #ArtificialIntelligence

9

Attached is NVDA's price chart for the last year - up c.175% (up 2500%+ in L5Y period!!) 🚀 I nonetheless have conviction that it currently remains undervalued... Among other things, due to strong competitive positioning as a GPU-maker, AI-driven demand fuelled by investor and consumer interest, and moats it has built. This is mirrored by equity analysts – GS’ latest price target was $800, BofA’s $1,500. I do believe the growth is capped, and driven in equal measure by under-supply and bubble-like demand as it is by fundamentals. Even so, Nvidia remains an attractive medium term investment. ⚖️ Demand – Supply Dynamics: AI is a generational innovation, and agents /apps are already upending SaaS, or at least so think VCs who deployed $64.6B in AI start-ups in 2023 alone. As early movers and now the industry standard (c.80% market share), Nvidia's A100 and H100 chips are not only the most efficient option, but also crucial for large scale AI training and inference (chips are the biggest cost head for LLM companies). With unprecedented interest in AI (ChatGPT acquired 1M users in 5 days of launch, OpenAI reported $3.4B ARR in Apr’24, AI agents are widely recognised as the tech of the future), Nvidia stands to capture a large portion of the $250-$500B data centre conversion market over the next 5 years​. This is proven by the $22.6B in data centre revenue Nvidia brought in during Q2’24, up 23% from the previous quarter and up 427% from a year ago 🥇Competitive positioning: While Intel, AMD, Arm have been active, the only credible competitor, AMD, has a relatively smaller market share in the high-performance AI GPU segment, and Nvidia’s consistent technological advancements keep it further ahead, e.g., Nvidia’s Rubin which will succeed Blackwell in 2026 is orders of magnitude better than anything on-market. 🤝Moats via ecosystem building: Collaborations with hyperscaler cloud providers like AWS and Microsoft Azure have expanded Nvidia’s reach. Moreover, acquisitions like Mellanox have strengthened its position in high-performance networking, critical to AI infrastructure. Coreweave, a prominent cloud services partner / distribution channel for chip capacity is an NVIDIA investee and is built almost entirely on Nvidia chips. Finally, they continue investing in networking, software tools, data centre solutions but also purpose-built AI PC and smartphones to create an ecosystem that not only locks in customers and enhances switching costs, but pivots more of their revenue to more predictable recurring contracts. 📈 Financial underpinnings: Nvidia’s expected FCF for next 2 years stands at $100B, providing huge ammo for R&D, strategic investments and M&A (only $30 to $35B earmarked for buybacks) and in turn, growth. Even so, their shares trade at a P/E ratio of c.20x  NTM earnings, far below historical median of 35x+, indicating the market expects them to grow in the medium term. KEEN TO HEAR THOUGHTS! #Nvidia #AI #Compute #Chips #Tech #Growth

10

1 Comment