Swami Sivasubramanian

👉 Amazon OpenSearch Ingestion adds support for ingesting data from self-managed sources Amazon OpenSearch Ingestion now allows you to ingest data from self-managed OpenSearch, Elasticsearch and Apache Kafka clusters, eliminating the need to run and manage 3rd party tools like Logstash to migrate your data from self-managed sources into Amazon OpenSearch Service. Now you can seamlessly migrate or continuously replicate your data from all OpenSearch versions and Elasticsearch 7.x versions either on Amazon EC2 or on-premises environments into Amazon OpenSearch Service managed clusters or Serverless collections. #aws #amazonopensearch #seamlessingestion

6

Very interesting post on Intuit's use of LLMs to manage K8s, via Aditya Kulkarni at InfoQ: "Intuit Engineering's Approach to Simplifying Kubernetes Management with AI" https://lnkd.in/eRN6_sAK

2

AWS continues to add capabilities to Amazon Bedrock that make it the easiest way to build high quality, low latency, and cost-effective GenAI applications. Today the team announced:   - Custom Model Import, which lets customers easily bring their own models (from SageMaker and S3) to Bedrock to leverage its leading GenAI features, such as RAG, agents, Knowledge Bases (guardrails), inference deployment, and advanced customization to keep teaching and refining models.   - Model Evaluation, which lets customers quickly select predefined evaluation criteria (like accuracy and robustness) and run reports that help them understand which is the best model for them.   - Meta’s new Llama 3 models, now available on Bedrock and can support a wide range of use cases including text summarization and classification, sentiment analysis, language translation, and code generation.   - New Cohere Models, Command R and R+, are state-of-the-art FMs customers can use to build enterprise-grade generative AI applications with advanced RAG capabilities, in 10 languages, to support their global business operations. These will arrive on Bedrock in the coming weeks.   - New Amazon Titan models with Amazon Titan Image Generator (GA today) helping customers produce studio-quality images, at low cost, using natural language prompts (and applies an invisible watermark to help reduce the spread of disinformation)… along with Amazon Titan Text Embeddings V2 model, which reduces storage and compute costs, while out-performing other leading models on text embeddings.   Tens of thousands of customers are already building real-world production-ready GenAI apps and we're excited for even more customers to get started. https://lnkd.in/g4rkak47

2,849

52 Comments

#Amazon #OpenSearch Ingestion now allows you to ingest streaming data from Amazon Managed Streaming for Apache Kafka (#MSK) Serverless, enabling you to seamlessly index the data from Amazon MSK Serverless clusters in Amazon OpenSearch Service managed clusters or #Serverless collections without the need for any third-party data connectors. #ApacheKafka #streamingdata #datastreaming #data https://lnkd.in/gdDJED6V

19

Heading to #reinvent2024 next month? Don’t miss our hands-on workshop on Dec. 4th on scaling generative AI workloads with Anyscale & Amazon Web Services (AWS) Hands-on session includes: 🔹 Building end-to-end LLM applications with RayTurbo, Anyscale's optimized Ray runtime. 🔹 Applying advanced performance techniques like dynamic autoscaling, heterogeneous hardware integration (CPU/GPU), and replica compaction. 🔹 Fine-tuning large language models with Anyscale’s LLM-Forge solution. 🔹 Using Anyscale observability tools to debug, diagnose, and profile distributed AI workloads effectively. ....and more! Register now: https://lnkd.in/gqRYFPz6 Seats are limited—sign up today!

9

Sometimes small changes are a big deal. S3's per-account bucket limit has meant that buckets have been more of a human provisioning thing, starkly different from objects, which tend to be interacted with primarily from code. For years, customers have been asking us to increase this limit -- and in the storage innovation talk at re:Invent this year, i'd tell a bit of a story about why it was a more complicated bit of work than you might expect. With this limit increase, buckets have become a much more programmatic construct. We hear customers talking about using buckets, with these limit increases, as a way of managing data sharing with their own customers and using them to more effectively isolate data for specific jobs and teams. If larger numbers of buckets opens up a new pattern for you, I'd love to hear more about it. These launches, like object consistency, S3 conditionals, and big bucket limit increases have a tendency to fly a little below radar. But they are the ones that I actually hear the most excitement back from developers about, because they really broaden S3 as a programmatic interface to your data, and a building block for really incredible systems. I'm so impressed with the hard and careful work on the team that went in to getting this one out. https://lnkd.in/gA2cWATM

245

8 Comments

When we started AWS, we looked at the undesirable situation of developers spending 70% of their time on the undifferentiated muck of managing infrastructure (vs only 30% on what differentiated their customer experiences) and aimed to flip that equation on its head. It’s made a pretty big difference in innovation and developer productivity.   Roughly 18 years later, we’re trying to solve another 70-30-like misalignment for developers, where developers are telling us that they’re spending roughly 70% of their time on repetitive and tedious tasks and code.   Today, we’re launching the general availability of Amazon Q, the world’s most capable GenAI-powered assistant for accelerating software development and leveraging companies’ internal data—and that aims to remove much of the muck of repetitive and tedious coding and data tasks for developers and employees at large.   On the software development side, Q doesn’t just generate code, it also tests code, debugs coding conflicts, and transforms code from one form to another (today, developers can save months using Q to move from older versions of Java to newer, more secure and capable ones; in the near future, Q will help developers transform their .net code as well). Q Agents does multi-step planning and reasoning to allow developers to string together multiple requests and have Q implement them.   On the internal data side, most companies have voluminous troves of internally relevant data that resides in wikis, intranet pages, Salesforce, storage repositories like Amazon S3, and a bevy of other data stores and SaaS apps that are hard to access. It makes answering straightforward questions about company policies, products, business results, code, people, and many other topics hard and frustrating. Q makes this much simpler. You can point Q at all of your enterprise data repositories, and it’ll search all this data, summarize logically, analyze trends, and engage in dialog with customers about it.   We also introduced today a powerful new capability called Q Apps, which lets employees describe, in natural language, what apps they want to build on top of this internal data, and Q Apps will quickly generate that app. This is going to make it *so* much easier for internal teams to build useful apps from their own data.   Customers are gravitating to Q, and we already see companies like Brightcove, British Telecom, Datadog, GitLab, GoDaddy, National Australia Bank, NCS, Netsmart, Slalom, Smartsheet, Sun Life, Tata Consultancy Services, Toyota, and Wiz using Q. And, we’ve only been in beta til today.   Very excited about how Q will change what’s possible for our customers, and being a part of helping them innovate more quickly.

9,816

170 Comments

Amazon MemoryDB: A fast and durable memory-first cloud database with Redis protocol support https://lnkd.in/eJUwwuR6

8

Here's a practical example of how Formal Methods can make code faster: AWS's new proven-correct authorization engine is 65% faster (even at p99.9) than the previous version. https://lnkd.in/gVCdurKh Sean says in the talk "we wouldn't have attempted a lot of optimizations we did without this backing of proof". Formal methods allow us to optimize systems more boldly, by removing the risk that optimizations will affect correctness. They also allow our teams to move faster, with lower risk, by building confidence in correctness early on in the software development process.

434

4 Comments

Coding is dead. Long live coding? Interesting take from AWS’s new CEO which just seems to echo what Jensen Huang from NVIDIA said a few months ago. What do you think of the idea that software engineers’ days are numbered because AI will just create what we want? AWS talk a lot about removing the “undifferentiated heavy lifting” from IT which basically is talking about the shared responsibility model where the cloud provider is responsible for the operation and security of the cloud while the customer is responsible for the operation and security of what they put on top of the cloud. Using more software-defined automation and higher level cloud services takes a lot of the manual effort out of IT, but it’s still IT. Software development has always been about reusing code and not unnecessarily reinventing the wheel each time. In my day, we had to have libraries of code, reusable functions you could include or you just searched the existing code for a comment that described something similar to what you wanted to do and copy the code. Without the internet that was largely limited to the code we wrote as a team internally. More recently, devs can Google, GitHub and Stack Overflow for code tutorials, functions, open source code and peer support from around the world. Generative AI just makes it easier to find a code pattern or best practice in almost any programming language, and embedding AI into IDEs saves you a click or three. I suppose it’s then natural to ask what’s the point of developers if they’re just using Gen AI to copy code patterns. Remove the middleman and just let AI do the job. This forgets that the most important quality of a developer is not their ability to write well structured code but to solve human problems with software. The tools we use to solve those problems keeps evolving over time, but the idea of AI coding the computers that code the AI in some kind of scene from the movie Inception is a bit bizarre. We still have low level languages today. Why? Because some use cases need that level of precision or efficiency, like embedded systems, high performance applications or parallel computing. So far high level programming languages haven’t displaced all the low level applications. Prompt engineering is just another form of high level abstraction that is interpreted by a LLM model, translated into executable code which crunches the numbers and produces a human readable result. Innovation in tech is about someone having a good idea while someone else finds a way to implement it 100 times faster or to make a ton more money out of it. It’s takes human imagination, ingenuity and problem solving - aided by technology - to keep the flywheel turning faster. I don’t think we’re quite ready for “just add AI” and watch the money pour in while we all kick back and sip cocktails by the beach. Not yet at least. https://lnkd.in/eKdW8-VE

5

1 Comment

Agents are a form of automation. But automation until now was a tedious undertaking where workflows needed to be mapped, business objectives needed to be understood and specific and deterministic outcomes needed to be guaranteed. Compared to LLMs and specifically abilities such as tool calling, the previous wave of automation can be called "shallow" intelligence. Shallow because of lack of autonomy. Intelligence (still) because it did improve efficiency. With GenAI, we have the potential to generate plans and workflows. This is a major difference from the previous wave of automation. Entire workflows can be generated by LLMs and executed by tools. To understand Agents, try to automate a workflow that you execute daily. A typical task of developing a comprehensive project plan is a great example. 1. Talk to the Business team. 2. Business Analyst maps various processes and business outcomes. 3. Product Manager converts details to a PRD. 4. Engineering interprets PRD. 5. PgM scopes projects and inter-dependencies. And so on. Each step is a combination of applying context and using a tool. A PM may use wiki to write a PRD and draw upon their knowledge of the product/service. In writing the PRD they may search across internal knowledge repositories, read other design documents and talk to stakeholders across the value chain. E.g., BusinessAnalystAgent: Transcribe meeting audio (TranscriptionTool_audioInput). Summarize the transcript (SummarizationTool_transcriptOutput). EngineeringContextAgent: Search the codebase for references (CodebaseSearchTool_PRDReferences). Analyze dependencies (DependencyAnalysisTool_codeDependencies). Address engineering questions (QAAgent_PRDQuestions). The above are components in a large workflow that can be executed by specific Agents with access to various tools. In a recent paper title DataLab: A Unifed Platform for LLM-Powered Business Intelligence (see attached image from the paper describing various modules in an Agentic BI workflow), the authors showcase in great details what the new form of automation in BI tasks may look like. For those of you seeking to learn about Agents, I would strongly recommend this paper. Start building a simple automation on popular datasets from Kaggle. Use frameworks like AutoGen or CrewAI and create simple tools that query databases, plot graphs and summarise tables. Try to build a system with various moving parts and multi-step workflows. The best way to learn about Agents is to automate one of your own day to day tasks. And with some quiet period coming up in a few weeks, you can even create Agents that will research gift ideas for your family and friends. I'll be looking to build a reading recommendation agent which takes details from a LinkedIn profile and creates a holiday special reading list. This list would be an eclectic collection of tweet threads, substacks, books, research papers and blog articles.

18

🚀 Want to run your very own trained LLM, privately and securely? 🤖🔒 Imagine harnessing the power of a Language Model tailored just for you without worrying about third-party servers! 💼✨ Try, LM Studio. Jest for few click . LLM running in your PC. With LM Studio, you can ... 🤖 • Run LLMs on your laptop, entirely offline 📚 • Chat with your local documents (new in 0.3) 👾 • Use models through the in-app Chat UI or an OpenAI compatible local server 📂 • Download any compatible model files from Hugging Face 🤗 repositories 🔭 • Discover new & noteworthy LLMs right inside the app's Discover page #LLM #private #chatgpt #AI

9

Distributed transactions can be hard. Reading this beautiful paper by Dynamo Db team https://lnkd.in/g8rXxVzT to learn how probably the world's biggest and busiest database handles this.

2

The Site Reliability Engineering: How Google Runs Production System book is arguably one of the most prevalent and well-known SRE resources out there, but there are many other great options for folks looking to expand on their learning. We recently spoke to Google's Reliability Advocate, Steve McGhee, in our Humans of Reliability interview series. In addition to his interesting anecdotes on the early days of SRE at Google, and his journey to becoming a Reliability Advocate, he also shared a handful of his favorite SRE resources, which we compiled here into a list: https://lnkd.in/gpUVhyRa

16

JFrog reported its quarterly earnings on Aug 7th. ProntoNLP compared the JFrog earning call transcript to 3 other similar SAAS (DevOps) companies as well as to the average of the software GICS group. ProntoNLP's Aspect based LLM signal gave an early warning for JFrog on May 9th 2024 (their last earning call) when the Aspects score dropped from 0.52 to 0.4. ProntoNLP analyzed the earning call transcripts of these 4 SAAS (DevOps) companies JFrog, Atlassian, Dynatrace and Datadog. As can be seen in the image below the only company that had an increase in the LLM score compared with its score last quarter is Atlassian while JFrog had a much lower LLM score (-16%) compared with their last quarter (Color Legend is at the top part of the image below). The dotted navy blue line shows the average LLM score for each of the last 4 quarters for the whole software GICS group. All these 4 companies are SAAS companies and hence the customer related LLM categories are extremely important. The image below shows that Datadog and Atlassian have extremely high values for Customer Acquisition, Customer Behavior and Customer Retention. ProntoNLP's new LLM Aspects based signal for these companies shows the following trend: JFrog: a huge decline from 0.4 last quarter to 0.08 Atlassian: score stayed the same at 0.43 Dynatrace: slight decline from 0.45 last quarter to 0.42 Datadog : slight decline from 0.47 last quarter to 0.42 Based on the LLM scores in the Customer related categories and the change in the Aspects scores of the 4 software stocks it seems that Atlassian and Datadog are the top picks now and JFrog is at the bottom of the pack. The new comparison feature of ProntoNLP is extremely powerful and can support picking the winners and losers within any GICS group based on the latest #GenerativeAI technologies. ProntoNLP provides via SFTP real time quantitative signals on earning calls, filings, macro and news. Reach out to ProntoNLP to start using our #GenerativeAI based tools and Maintain an edge in alpha creation. #Quant #HedgeFunds #ArtificialIntelligence #StockMarket

7

This is incredible news for Amazon Kinesis Data Streams On-Demand users! KDS On-Demand now allows writing up to 10 GB/s, a significant 5x enhancement! The most remarkable aspect is that there is no action required to leverage this feature. #Kinesis #data #BigData #streamingdata #Datastreaming #ondemand #AWS Link: https://lnkd.in/gdm_FXrU

47

2 Comments

Fun facts: 1) Ben Treynor talked to me about STPA and that conversation was one of the reasons I decided to come to Google. (I didn’t join Google - I joined Ben. Had I known that I would also get to work with Tim, that would have been another reason to join!) 2) We’ve been working with John Thomas to apply/tweak/apply STPA to complicated software systems. He his fantastic! 3) STPA is NOT a ‘read the book and give it a shot’ sort of thing. If you want to get STPA going in your org, you should connect with John to figure out how.

110

Will be really interesting to check how mixed Agentforce agents for Sales, Service, Marketing, Commerce, etc. work together that would balance the utilization of cloud computing resources, or instead influence cloud providers’ computing, memory, networking resource configurations.

16

1 Comment