Murat Akbacak

The 7th workshop on Neural Scaling Laws is about to start: https://lnkd.in/gQSB5k8v Scaling laws, emergency, alignment, transfer learning, with a focused track on multilingual models. Looking forward to kick it off with Irina Rish and discuss all these topics with Yuhai Tu, Paul Bogdan, Hailey Schoelkopf, Darshil Doshi, Rio Yokota, Neha Sengupta, Preslav Nakov, Tatiana Shavrina, PhD, Yishi Xu. Join in person or virtually! #icml2024 #ai

37

Ladies and Gentlemen, here is the new paper from Meta: LCM, Large Convept Models, no longer token predictors but concept predictors! 💡As in the human brain🧠we do not reason by building sentences, word by word, but we start from abstract concepts, ideas that combine without essentially resorting to language!

7

NVIDIA Nemotron might not be the open weight, small size, state-of-the-art model that everyone was waiting for, but it is a groundbreaking advance in open weight models. My prediction is that by Q1 2025, we should have an open weight model rivalling current state-of-the-art models by OpenAI and Anthropic. We broadly understand how to get there.

1

https://lnkd.in/eXbCsAnD “He who controls the GPUs, controls the universe.” Access to GPUs, and in particular those made by Nvidia, the leading supplier, is vital for any company that wants to be taken seriously in #ai. Nvidia’s dominance has driven its market value to more than $2trn. 'GPUs do the computational heavy-lifting needed to train & operate large #ai models. Yet, oddly, this is not what they were designed for. The initials stand for “graphics processing unit”, because such chips were originally designed to process video-game graphics. It turned out that, fortunately for Nvidia, they could be repurposed for #ai workloads. 'Might it be better to design specialist ai chips from scratch? That is what many companies, small & large, are now doing in a bid to topple Nvidia. Dedicated ai chips promise to make building and running ai models faster, cheaper or both. Any firm that can mount a credible threat will have no shortage of customers, who dislike its lofty prices and limited supplies. 'GPUs contain several thousand processing engines, or “cores”, which let them run thousands of versions of the same simple task at the same time. Running #ai models similarly involves running lots of copies of the same task in parallel. Figuring out how to rewrite ai code to run on GPUs was one of the factors that triggered the current ai boom. 'Yet GPUs have their limitations, particularly when it comes to the speed with which data can be shuffled on and off them... When training very large #ai models, some GPu cores may be idle as much as half of the time. Andrew Feldman, the boss of Cerebras, a startup based in Sunnyvale, CA, likens it to the gridlock in a grocery store on the day before Thanksgiving. “Everybody’s in a queue, so there are blockages in the parking lot, there are blockages in the aisles, blockages at the checkout. That’s exactly what’s happening with a gpu.” 'Cerebras’s response is to put 900,000 cores, plus lots of memory, onto a single, enormous chip, to reduce the complexity of connecting up multiple chips & piping data between them. 'Groq, a startup from Mountain View, CA, is taking a different approach. Its ai chips, called language processing units (lpus), are optimised to run #llms particularly quickly. 'Yet another approach is that taken by MatX, also based in CA. GPUs contain features & circuitry that provide flexibility for graphics, but are not needed for llms. The firm is working on getting rid of such unnecessary cruft.

14

3 Comments

Smaller #genAI models are changing the game! Introducing #mistral-NeMo by #nvidia and #mistralAI with 128k token context window - a tool for businesses that can run on RTX GPUs which you may have at your home. Super inexpensive with no high cloud costs, keeps data private on your private computer! #nvidia #mistralai #genaiforenterprise #responsibleai

10

Emerging technologies don’t simply materialize fully formed; they require the establishment of supply chains, the development of manufacturing processes, and the evolution of the technology itself. Along this journey, the cost incurred includes learning from the ten unsuccessful paths before arriving at the eleventh one that proves effective…a perceptive read if you are working in the Spatial Computing space

20

The future of how we will operate startups / businesses - with multi agent collaboration on different business departments. #multiagent #llm #ai #startups #business #future

5

What an end to year 2024, OpenAI o3 has achieved state of the art results on ARC-AGI benchmarks. From ARC-AGI's blog (https://lnkd.in/gKw3bU4U): "To sum up – o3 represents a significant leap forward. Its performance on ARC-AGI highlights a genuine breakthrough in adaptability and generalization, in a way that no other benchmark could have made as explicit. o3 fixes the fundamental limitation of the LLM paradigm – the inability to recombine knowledge at test time – and it does so via a form of LLM-guided natural language program search. This is not just incremental progress; it is new territory, and it demands serious scientific attention." This is indeed a significant leap and in 2025, I expect few things: * OpenAI, Google and others will keep pushing the boundaries with what we can do with reasoning models, models will keep getting better - we have not yet reached the ceiling. * We will see openweights models as well which will be comparable to o3 (we already have few going in that direction like QwQ, Deepseek - both from China). * Products will make an accelerated effort for catching up both of these: https://lnkd.in/gRBuFMGi

45

Our #icra2024 paper is now available from #ieee dl. We proposed a new Inverse Reinforcement Learning based method for hand trajectory prediction. Mukund Mitra Gyanig K. Partha Pratim Chakrabarti

18

2 Comments

‘I Hope Suffering Happens To You’: Nvidia CEO’s Case For Hiring For Grit Over Pedigree by Jack Kelly Forbes “I wish you ample doses of pain and suffering.” That is the unconventional message NVIDIA CEO and founder Jensen Huang had for students at Stanford University, his alma mater, where he spoke last month at the Stanford Institute for Economic Policy Research. According to Huang, elite educational pedigrees alone are not necessarily the best predictor of success. “Greatness is not intelligence. Greatness comes from character. And character isn’t formed out of smart people, it’s formed out of people who suffered,” said the Stanford alum. Instead, the man who presides over one the world’s most valuable companies in terms of stock market valuation, now looks to find certain traits in job applicants that include resilience, grit and determination, which he contends lead to innovation and success in the workplace. Counterintuitively, Huang believes that having very high expectations coming out of an Ivy League education can potentially set people up for less resilience when faced with failure or adversity, which are inevitable in any career path. “Unfortunately, resilience matters in success,” he said to the students. “I don’t know how to teach it to you except for I hope suffering happens to you.” His comments indicate a view that real-world experience, involving struggles, obstacles and suffering, can instill the resilience needed for long-term success. Therefore, he believes that employers should look beyond just the prestige factor of a university and more closely at the inner drive, mindset and resolution of potential hires. “To this day I use the phrase ‘pain and suffering’ inside our company with great glee,” said the chief executive. “I mean that in a happy way, because you want to refine the character of your company. You want greatness out of them.” Hiring PHDs The Nvidia founder’s statements about valuing resilience over educational pedigrees align with the old Wall Street adage about hiring what they call “PHDs”—the poor, hungry and driven. The core parallels are prioritizing hunger, grit and motivation over academic credentials alone, believing that candidates who have had to overcome adversity, financial hardships or personal trauma offer invaluable drive and tenacity. The PHD concept essentially argues that struggle itself, if handled constructively, can be great preparation for thriving in ultra-competitive business environments where stamina and an insatiable work ethic are keys to success. A person who had to fight for opportunities may have a stronger inner drive and motivation to succeed compared to ... https://lnkd.in/e5XZitzS

4

2 Comments

🌟 Exploring the Intersection of Business and Technology re AI! 🌟 Before my decade-long focus on AI, connecting technology and business in various domains has always been my passion; This is why I enjoyed this conversation with David Luan - No BS, No Twitter exaggerations, No clickbate - Just a super well informed, super smart David sharing his insights from years of hands-on experience: #Startegy, #Software, #Hardware, #Platforms, #ModelSpace and everything in between - LMK what you think of his projections. In This Episode: 1. The Biggest Lessons from OpenAI and Google Brain: What did OpenAI realize that no one else did that allowed them to steal the show with ChatGPT? Why did it take 6 years after the introduction of transformers for ChatGPT to be released? What are 1-2 of David’s biggest lessons from his time leading teams at OpenAI and Google Brain? 2. Foundation Models: The Hard Truths: Why does David strongly disagree that the performance of foundation models is at a stage of diminishing returns? Why does David believe there will only be 5-7 foundation model providers? What will separate those who win vs those who do not? Does David believe we are seeing the commoditization of foundation models? How and when will we solve core problems of both reasoning and memory for foundation models? 3. Bunding vs Unbundling: Why Chips Are Coming for Models: Why does David believe that Jensen and Nvidia have to move into the model layer to sustain their competitive advantage? Why does David believe that the largest model providers have to make their own chips to make their business model sustainable? What does David believe is the future of the chip and infrastructure layer? 4. The Application Layer: Why Everyone Will Have an Agent: What is the difference between traditional RPA vs agents? Why are agents a 1,000x larger business than RPA? In a world where everyone has an agent, what does the future of work look like? Why does David disagree with the notion of “selling the work” and not the tool? What is the business model for the next generation of application-layer AI companies? https://lnkd.in/dUpWWzvN

4

The post below addresses today's realities, but let's not overlook the future possibilities. The AI hype should be viewed as an opportunity for learning and course correction. However, expectations often don't meet the hype. It's essential to recognize that improvements are emerging rapidly, and we must keep pace. Brouhaha about AI issues in large scheme of things should be taken as learnings and course correction. Everyone wants to leverage AI to avoid being labeled "obsolete" and also to gain recognition. The numerous choices in the AI landscape can be overwhelming. Model garden is now like a forest now. Hyperscalers are wooing subscribers, to get on board like “early bird catches the worm” syndrome. They are set up like dealerships to make some quick buck on them before the gullibles realize what they have is a Rubik’s cube which only few are able to solve at first. On the other hand, the open-source community is at the crossroads of innovation and accountability with the California senate bill 1047, which aims to impose requirements on developers of advanced AI models. Solution architects are caught up in the web of offerings where each one touts it is better than other and cheaper too. Agentic models, MoE (Mixture of Experts) , Improved RAG/ Graph-RAG with integration of multi-model retrieval techniques , larger context tokens and longer context retention and models that have State space model, MoE and transformers all combined into one. Use cases are plenty and it is a climb up the mountain and not a glide as it takes time and patience. At this moment many companies are witnessing benefits in efficiency and productivity while giving time back to humans to focus on creative and innovative ways to improve their workplace and life. Something that had become clerical, boring and no-brainer has been delegated to these AI bots ( what I would loosely refer to as RPA 2.0 at the risk of being blasphemous). While some companies have rushed in with “fail-fast” concept to the detriment of the end consumer. Product managers are grappling with use cases to incorporate AI into their system for the fear of being called “un-imaginative” or not in step with the times. High school students are building, creating, innovating and publishing papers using these technologies. Nevertheless, the ecosystem is converging towards a shared goal: creating a universe of interconnected applications. This is a collective effort, driven by passion and fueled by technology, ideas, and people coming together at unprecedented levels.

14

1 Comment

Transformers Transformers has been a major milestone in the field of NLP and heavily used in Generative AI. There are 3 variants of transformer based models and they are 1. Encoder-only 2. Decoder-only 3. Encoder-Decoder Encoder only models: These are also called as autoencoders and these are pretrained using a technique called masked language modeling. A text with a random masked token is sent to the model to predict the masked token. For example, consider the text "If you don't stop at the sign, you will get a ticket" The training input that will be passed on the encoder model is "If you don't _____ at the sign, you will get a ticket. The model is expected to predict the token (word) "stop" These models use bi-directional representations of the input to better understand the full context of a token. Examples of encoder models: BERT Family models Decoder only models: These models are called autoregressive models and are pretrained using a technique called causal language modeling. These models predict the next token using the previous tokens. For example, consider the text "If you don't stop at the sign, you will get a ticket" The training input that will be passed on the encoder model is "If you don't ______" The model will still try to predict the word "stop", but only based on previous tokens. These models are used for generative tasks, including question answering. Examples of decoder only models: GPT Family, Falcon, LLama models Encoder decoder models: These models are called sequence to sequence models and the pretraining objective varies from model to model. For example the popular FLAN - T5 uses a consecutive multitoken masking called span corruption. For example, consider the text "If you don't stop at the sign, you will get a ticket" The training input that will be passed on the encoder model is "If you don't _____ _______ the sign, you will get a ticket. The model will try to predict the tokens "stop at" These models are good at translation tasks. Examples: T5 family In all the above explanations we just took one sentence of text. The LLMs you see in the market are trained on huge volumes of text available over the internet. #LLM #encoder #decoder #transformer #genai

3

Apple announces new iPads with more AI and all I really care about in this is more CPU and GPU. i’m still not sure what the practical point of AI on device is, but I’m sure that Apple will make that case at WWDC. ——- ArsTechnica: “Apple announces M4 with more CPU cores and AI focus, just months after M3. Aggressive update schedule is a major departure for Apple Silicon.” “In a major shake-up of its chip roadmap, Apple has announced a new M4 processor for today’s iPad Pro refresh, barely six months after releasing the first MacBook Pros with the M3 and not even two months after updating the MacBook Air with the M3. Apple says the M4 includes "up to" four high-performance CPU cores, six high-efficiency cores, and a 10-core GPU. Apple's high-level performance estimates say that the M4 has 50 percent faster CPU performance and four times as much graphics performance. Like the GPU in the M3, the M4 also supports hardware-accelerated ray-tracing to enable more advanced lighting effects in games and other apps. Due partly to its "second-generation" 3 nm manufacturing process, Apple says the M4 can match the performance of the M2 while using just half the power. ArsTechnica: https://lnkd.in/g7ypBZ_U ——- ArsTechnica: “New iPad Pros are the thinnest Apple device ever, feature dual-OLED screens. They also contain what Apple calls the fastest consumer AI computer you can buy.” “Apple's newest iPad Pro puts an M4 chip inside a thinner frame and is available in new 11-inch and 13-inch sizes, while also upgrading the screens on both to "tandem" OLED displays for more brightness. Compared to the last iPad Pro, released in early 2022, Apple is highlighting how thin and light these new Pros are. The 11-inch model is 5.3 mm thick and weighs less than a pound, while the 13-inch is 5.1 mm, which Apple says is its thinnest product ever, at 1.28 pounds.” ArsTechnica: https://lnkd.in/gX8Qipwz

1

1 Comment

Learn how developers are using generative AI, NVIDIA NIM microservices, NVIDIA Omniverse and Universal Scene Description (OpenUSD) to build applications and workflows that enable brand-accurate, targeted and efficient advertising at scale.

6

1 Comment

Meet AGUVIS: A pure vision-based framework for autonomous Computer/Mobile-use GUI agents, operating seamlessly across web, desktop, and mobile platforms without UI code. Key Features & Contributions - Pure Vision Framework: First fully autonomous pure vision GUI agent capable of performing tasks independently without relying on closed-source models - Cross-Platform Unification: Unified action space and plugin system that works consistently across different GUI environments - Comprehensive Dataset: Large-scale dataset of GUI agent trajectories with multimodal grounding and reasoning - Two-Stage Training: Novel training pipeline focusing on GUI grounding followed by planning and reasoning - Inner Monologue: Explicit planning and reasoning capabilities integrated into the model training - Generalization to OOD (Out-of-Distribution): AGUVIS demonstrates impressive generalization on OSWorld benchmark even without desktop trajectory data during training. Project Page: https://lnkd.in/guzkwpbB Paper: https://lnkd.in/gs7wXyJS GitHub: https://lnkd.in/gWeSkFmK Collaboratiion among Yiheng Xu Zekun Wang Junli Wang Dunjie Lu Tianbao Xie Amrita Saha Doyen Sahoo Tao Yu Caiming Xiong

156

6 Comments

Arm is upstreaming #Kleidi technology into #PyTorch with amazing effect - up to 3x faster CPU #inference in Eager mode, and up to 2x in Graph mode. These optimizations make running #LLMs on CPU more practical than ever. Check out our blog for more details! #Arm #Neoverse #Graviton https://lnkd.in/gvrh67v2

17

*The 10 Commandments of AI Agent User Experience* With all the hype around AI agents and automation, it's easy to forget the most critical element: there's always a human in the loop. Without a pleasant user experience, even the best agent won't get far—the users will shut it down. So, if you want your AI agent to succeed, here are 10 commandments to build it right—for humans. 1. Necessity: Create an AI agent only if it adds value 2. Transparency: Always indicate it's an AI agent 3.  Information: Show status to build trust during wait times 4.  Escalation: Know when to involve humans 5. Visibility: Let users retrace (and revert) agent steps 6. Balance: Allocate tasks wisely between human and agent 7. Personalization: Adjust the tone & allow customization to user preferences 8. Multimodality: Use different modes appropriately 9. Feedback: Continuously learn from users and improve 10. Accessibility: Ensure usability for all users What commandments would you add to the list? Let’s keep the conversation going!

31

2 Comments

Triple arxiv drop from the last couple of weeks: Remember when we were excited with snapfusion running diffusion on device? we now do diffusion-based super res on 4k x 3k images on the phone. And still match the SOTA results. This is how: Edge-SD-SR: Low Latency and Parameter Efficient On-device Super-Resolution with Stable Diffusion via Bidirectional Conditioning https://lnkd.in/e6DD-bJG CLIP models struggle with compositionally and manifest strong bag-of-words behaviour. Meanwhile, while benefiting from strong language understanding capabilities, Visual LLMs trail behind contrastively trained models for discriminative tasks like retrieval. We propose a novel training method that enhances the Visual LLMs retrieval ability, achieving SOTA performance for 1-stage models on discriminative tasks. Discriminative Fine-tuning of LVLMs https://lnkd.in/eS4syJvT Want to create high-resolution images but your diffusion model is trained to operate on lower resolutions? We propose a training-free method that does exactly that. And get state-of-the-art results among competitors. And with minimal latency overhead (much lower than most competing methods). Check it out here: FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion https://lnkd.in/eJVEkBVx

55

3 Comments