We’re introducing Llama 4 Scout and Llama 4 Maverick, the first open-weight natively multimodal models with unprecedented context length support and our first built using a mixture-of-experts (MoE) architecture. We’re also previewing Llama 4 Behemoth, one of the smartest LLMs in the world and our most powerful yet to serve as a teacher for our new models.

Llama 4 Scout, a 17 billion active parameter model with 16 experts, is the best multimodal model in the world in its class and is more powerful than all previous generation Llama models, while fitting in a single NVIDIA H100 GPU. Additionally, Llama 4 Scout offers an industry-leading context window of 10M and delivers better results than Gemma 3

Llama 4 Maverick, a 17 billion active parameter model with 128 experts, is the best multimodal model in its class, beating GPT-4o and Gemini 2.0 Flash across a broad range of widely reported benchmarks

After tinkering with Mistral Small 3.1, I'm excited to try Llama 4 Scout once it lands in GitHub Models and Ollama.

I don't think that Mistral Small 3.1 uses MoE architecture which should help a ton considering the 10 million token context size.

Thanks to Gen Z, the site has found new life. As of 2025, Gen Z makes up 50% of Tumblr’s active monthly users and accounts for 60% of new sign-ups

Don't call it a comeback.

User numbers spiked in January during the near-ban of TikTok and jumped again last year when Brazil temporarily banned X.

To keep up with the momentum, Tumblr introduced Reddit-style Communities in December, letting users connect over shared interests like photography and video games. In January, it debuted Tumblr TV—a TikTok-like feature that serves as both a GIF search engine and a short-form video platform.

But perhaps Tumblr’s greatest strength is that it isn’t TikTok or Facebook.

For its users...that’s part of the appeal.

Tumblr's been through a rough ride, but at the core there's a lot of nice things to like about it. In previous posts, I talked about some opportunities for Tumblr as a result of replatforming on top of WordPress and its expressed intent for ActivityPub integration. In doing so, not only would it enable an intuitive front-end for publishing different kinds of posts, but it would also provide opportunities for self-hosting and a more federated and decentralized social platform.

I'm excited to see where this goes.

Related post from Business Insider's Amanda Hoover on the topic.

Awesome post from my friend maho.dev

I’ve introduced customizable templates for note generation. This enhancement allows the full content of a blog post to be included directly in ActivityPub notes, moving beyond mere link sharing—a practice often associated with bots—and leveraging the full potential of the Fediverse.

This is cool. I'd like something better for my POSSE setup. Might give this a try.

Today, we're introducing Gemma 3, a collection of lightweight, state-of-the-art open models built from the same research and technology that powers our Gemini 2.0 models. These are our most advanced, portable and responsibly developed open models yet. They are designed to run fast, directly on devices — from phones and laptops to workstations — helping developers create AI applications, wherever people need them. Gemma 3 comes in a range of sizes (1B, 4B, 12B and 27B), allowing you to choose the best model for your specific hardware and performance needs.

Build with the world's best single-accelerator model: Gemma 3 delivers state-of-the-art performance for its size, outperforming Llama3-405B, DeepSeek-V3 and o3-mini in preliminary human preference evaluations on LMArena’s leaderboard. This helps you to create engaging user experiences that can fit on a single GPU or TPU host.

Create AI with advanced text and visual reasoning capabilities: Easily build applications that analyze images, text, and short videos, opening up new possibilities for interactive and intelligent applications.

Handle complex tasks with an expanded context window: Gemma 3 offers a 128k-token context window to let your applications process and understand vast amounts of information.

Create AI-driven workflows using function calling: Gemma 3 supports function calling and structured output to help you automate tasks and build agentic experiences.

Today, we’re introducing Command A, a new state-of-the-art generative model optimized for demanding enterprises that require fast, secure, and high-quality AI. Command A delivers maximum performance with minimal hardware costs when compared to leading proprietary and open-weights models, such as GPT-4o and DeepSeek-V3. For private deployments, Command A excels on business-critical agentic and multilingual tasks, while being deployable on just two GPUs, compared to other models that typically require as many as 32.

Its 256k context length (2x most leading models) can handle much longer enterprise documents. Other key features include Cohere’s advanced retrieval-augmented generation (RAG) with verifiable citations, agentic tool use, enterprise-grade security, and strong multilingual performance.

The next generation of Cohere models will help power a range of AI applications for customers across industries like finance, healthcare, manufacturing, energy, and the public sector. In particular, they will seamlessly integrate with North, our secure AI agents platform to unlock the full potential of your company data and people with AI agents.

The tools on tools.simonwillison.net were mostly built using AI-assisted programming.

This page lists the commit messages for each tool, many of which link to the LLM transcript used to produce the code.

The descriptions for each of the tools were generated using Claude 3.7 Sonnet.

This is such a cool use of a website. Using AI, Simon has generated a set of small utilities that are useful to him. If they're useful to others though, he's made them avaiable on his website.

Building on Mistral Small 3, this new model comes with improved text performance, multimodal understanding, and an expanded context window of up to 128k tokens. The model outperforms comparable models like Gemma 3 and GPT-4o Mini, while delivering inference speeds of 150 tokens per second.

Lightweight: Mistral Small 3.1 can run on a single RTX 4090 or a Mac with 32GB RAM. This makes it a great fit for on-device use cases.

Just pre-ordered my Pebble Core Time 2 Duo! Sadly it won't be here until December.

Go to https://store.repebble.com/ to get yours.

Interesting post from Matt Biilman.

...the largest disruption from the current evolution of AI will come from bringing agency to computers.

Computers will perceive their environment and take actions autonomously in order to achieve goals.

Companies discovered that improving the DX of their products would empower and incentivize developers to extend their product with new capabilities and lead to huge competitive advantages. For developer tool companies, DX became a key competitive differentiator.

We need to start focusing on AX or “agent experience” — the holistic experience AI agents will have as the user of a product or platform.

Too many companies are focusing on adding shallow AI features all over their products or building yet another AI agent. The real breakthrough will be thinking about how your customers’ favorite agents can help them derive more value from your product. This requires thinking deeply about agents as a persona your team is building and developing for.

As AI agents start becoming useful and commonplace, we’re broadly going to see two approaches enabling agents to interact with the software we depend on:

A closed vertical approach, where companies tightly integrate their own agents into their own software. An open approach, where companies focus on making their software accessible to external agents.

Leaning into AX as a strategy, means embracing a vision of an open agent world. This vision aligns with the original ethos of the open web: a place where many diverse competing agents (built by different people or companies) can seamlessly interact with software on behalf of their users. Prioritizing AX makes it as simple as possible for any agent a user prefers, to deliver outcomes on their behalf.

Software designed for AI agents has the potential to deliver exponential value. As an industry we must collectively focus on building an open agent ecosystem and designing thoughtful AX to create a better, more open, and connected digital world.

We are announcing the Mercury family of diffusion large language models (dLLMs), a new generation of LLMs that push the frontier of fast, high-quality text generation.

Mercury is up to 10x faster than frontier speed-optimized LLMs. Our models run at over 1000 tokens/sec on NVIDIA H100s, a speed previously possible only using custom chips.

This is just a place for me to share. I been wanting a lil blog for years. Somewhere to post random shit I fuck with where the audience is way smaller than it is on the social media platforms. Finally pulled the trigger, bare with us as we still developing this page and the layout.

This is cool!

I ran into this while reading the post Blogging for smaller audiences and deeper connections by Herbert Lui.

Looking at other posts, it looks like he or at least someone is actively posting.

While going through a backlog of personal blogs I haven't read in a few weeks, I'm noticing a trend in posts that express similar sentiments to those in Naz's post.

I'm happy to see more people choosing to have a place that gives them more autonomy and freedom to express themselves in whichever form makes the most sense for them.

Your site is a home.

Eventually, social networks were created...novelty and the promise of interconnectedness by gathering in a common town square to blast out whatever was going on in our lives eventually won out.

But you might have still had your website, your home to return to...the place you built.

Things started to change and instead of going home, you, and everybody else started to live in the town square.

There is a shape to your physical home. Arranged and organized in the ways that make sense for you, the reward is a space that works for you.

A platform or network doesn’t allow for much configuration. The town square isn’t owned by you.

People became absolutely reliant on the same gathering place...the convenience of seeing friends (sometimes) outweighed your other neighbors spouting garbage and hate. You came to rely on this place for everything.

You can still have a home.

I have very little at the town square, because it’s not a public one. It’s a walled-off town square, whose rules and borders change at the whims of those who created it.

Thanks for visiting my home. I’m glad you dropped by.

I’d love to see yours sometime.

In this guide, we will set up a multistreaming workflow using Azure, an Azure VM, MonaServer 2, and FFmpeg to broadcast to multiple platforms like LinkedIn Live and YouTube Live. We’ll use OBS Studio as the main streaming software. Let’s get started.

Love this detailed writeup from Maho.

When I originally experimented with something similar using my Owncast instance, I was running ffmpeg locally but I like his solution better.

Unplatform is an interactive guidebook, online library, and recommendations database intended to help you escape social media and join the indie web.

Love this guide.

Test-time scaling is a promising new approach to language modeling that uses extra test-time compute to improve performance. Recently, OpenAI's o1 model showed this capability but did not publicly share its methodology, leading to many replication efforts. We seek the simplest approach to achieve test-time scaling and strong reasoning performance. First, we curate a small dataset s1K of 1,000 questions paired with reasoning traces relying on three criteria we validate through ablations: difficulty, diversity, and quality. Second, we develop budget forcing to control test-time compute by forcefully terminating the model's thinking process or lengthening it by appending "Wait" multiple times to the model's generation when it tries to end. This can lead the model to double-check its answer, often fixing incorrect reasoning steps. After supervised finetuning the Qwen2.5-32B-Instruct language model on s1K and equipping it with budget forcing, our model s1-32B exceeds o1-preview on competition math questions by up to 27% (MATH and AIME24). Further, scaling s1-32B with budget forcing allows extrapolating beyond its performance without test-time intervention: from 50% to 57% on AIME24. Our model, data, and code are open-source at this https URL

Today we're excited to announce Nomic Embed Text V2, our next-generation embedding model that brings the Mixture of Experts (MoE) architecture to text embeddings on a new expanded multilingual training dataset.

Personally, I found the part on MoE to be the most interesting about this release.

Rather than a dense model which uses all parameters on an input, the MoE architecture dynamically routes to different "experts" - sparse subsets of parameters at each layer - activating, ideally, only the parameters especially needed to process the input. This approach allows for more efficient use of compute when generating embeddings.

In our experiments, we found that alternating MoE layers with 8 experts and top-2 routing provides the optimal balance between performance and efficiency. This results in 475M total parameters in the model, but only 305M active during training and inference.

Research into embedding model architecture has significant practical implications for working with text embeddings in production:

• Lower latency for high-volume applications of embeddings like retrieval
  
  
• Reduced deployment costs through more efficient parameter usage
  
  
• More accessibility to embeddings in settings with constrained compute

Today, we’re announcing Claude 3.7 Sonnet1, our most intelligent model to date and the first hybrid reasoning model on the market. Claude 3.7 Sonnet can produce near-instant responses or extended, step-by-step thinking that is made visible to the user. API users also have fine-grained control over how long the model can think for.
Claude 3.7 Sonnet shows particularly strong improvements in coding and front-end web development. Along with the model, we’re also introducing a command line tool for agentic coding, Claude Code. Claude Code is available as a limited research preview, and enables developers to delegate substantial engineering tasks to Claude directly from their terminal.

Claude 3.7 Sonnet and Claude Code mark an important step towards AI systems that can truly augment human capabilities. With their ability to reason deeply, work autonomously, and collaborate effectively, they bring us closer to a future where AI enriches and expands what humans can achieve.

Gold.

All the techniques we'll cover in this book tackle one or several of the following three key challenges, which we'll keep bumping into throughout the book:

---

1. **Memory Usage**: it's a hard limitation - if a training step doesn't fit in memory, training cannot proceed 2. **Compute Efficiency**: we want our hardware to spend most time computing, so we need to reduce time spent on data transfers or waiting for other GPUs to perform work. 3. **Communication overhead**: we want to minimize communication overhead as it keeps GPUs idle. To archieve this we will try to make best use of intra-node (fast) and inter-node (slower) bandwidths as well as overlap communication with compute as much as possible.

---

In many places we'll see that we can trade one of these (computation, communication, memory) for another (e.g. recomputation or Tensor Parallelism). Finding the right balance is key to scaling training.

I didn't watch the Super Bowl this year so I didn't get a chance to watch the ads until a few days later.

From the ones I saw, I think Google's was my favorite.

Apart from the touching story, what stood out to me the most was its focus on humanity, with technology playing a supporting role rather than being the main character.

Great session on Tensors in .NET by Tanner

Tensors are the foundational data structure powering man AI workloads today.

Introduced in .NET 9, Tensors builds on top of earlier work like TensorPrimitives, hardware intrinsics, and generic math.

You can check out the recording below to learn more.

If you're a .NET developer looking to get started with Generative AI, there's a new course for beginners that just launched.

Through a series of self-paced lessons, you'll learn the foundations to help you start using AI to augment the capabilities of your .NET applications.

Check out the course and give us feedback!

So excited to see the AI Dev Gallery is now in the Microsoft Store.

Through various interactive samples, developers get to see how they might use AI for different tasks like text classification, object detection, and many others.

Best of all, the models are all running locally and you get to see and export the C# code powering the samples to Visual Studio so you can continue tinkering on your own and integrate into your own applications.

A few months ago, the team came on the .NET AI Community Standup to showcase the app. Since then, it's only kept improving and introducing new scenarios.

You can check out the recording from that stream here.

This free course will take you on a journey, from beginner to expert, in understanding, using and building AI agents.

At the end of this course you’ll understand how Agents work and how to build your own Agents using the latest libraries and tools.

There is a hidden value in blogging... You do it not for the applause but because it needs doing.

You write because you think, because you observe, because you need to put it somewhere.

If someone reads it? Bonus. If not? The work still got done.

💯💯💯

I agree with many of the points in Andy's post.

I don't have analytics on my website, so I couldn't tell you who or how many people are visiting it.

And that's okay.

When someone links to content they found on my website and tags me or sends me an e-mail letting me know something on my website is broken is when I know there's at least one person.

Many of the content on here serves as a reminder of how I've solved problems for myself. The microposts and responses are observations that capture a moment in time. If the messages reach someone on the other end and either solve their problem or resonate with them, I'm happy that happens, but I see that as a bonus.

TL;DR We’re making a new Pebble-style smartwatch.

This is amazing news! By the time I decided to get a Pebble, they'd already been sold to Google.

I agree with Eric's points here:

I've tried every single smart watch out there, but none do it for me. No one makes a smartwatch with the core set of features I want:

• Always-on e-paper screen (it’s reflective rather than emissive. Sunlight readable. Glanceable. Not distracting to others like a bright wrist)
• Long battery life (one less thing to charge. It’s annoying to need extra cables when traveling) Simple and beautiful user experience around a core set of features I use regularly (telling time, notifications, music control, alarms, weather, calendar, sleep/step tracking)
• Buttons! (to play/pause/skip music on my phone without looking at the screen)
• Hackable (apparently you can’t even write your own watchfaces for Apple Watch? That is wild. There were >16k watchfaces on the Pebble appstore!)

Garmin is the closest I've come to it at least on the battery life and buttons front. Pebble was one-of-a-kind.

I already signed up to stay up to date with the project and plan on being a day one customer. You can too at repebble.com.

P.S. While not surprised, I didn't know Eric was behind another amazing project I'm a fan of, Beeper.

Great to see more people sharing knowledge and linking to others from their own websites / platforms.

I decided to follow simon's approach to creating a link blog, where I can share interesting links I find on the internet along with my own comments and thoughts about them.

...this is an excellent way to share knowledge while also keeping a personal record. It’s much better than simply saving something to Readwise to read later and leaving a few highlights or dull comments like "interesting."

😂😂😂

This website is amazing.

We already finalized our OKRs for H1 but if you throw it in the backlog maybe we can get it prioritized in H2 planning

This could be part of a larger initiative, but let’s hold off for now

We should validate that with users first

Just like most existing LLMs, DeepSeek-R1 generates one token at a time, except it excels at solving math and reasoning problems because it is able to spend more time processing a problem through the process of generating thinking tokens that explain its chain of thought.

An agent that uses reasoning to synthesize large amounts of online information and complete multi-step research tasks for you.

Powered by a version of the upcoming OpenAI o3 model that’s optimized for web browsing and data analysis, it leverages reasoning to search, interpret, and analyze massive amounts of text, images, and PDFs on the internet, pivoting as needed in reaction to information it encounters.

How it works

Deep research was trained using end-to-end reinforcement learning on hard browsing and reasoning tasks across a range of domains. Through that training, it learned to plan and execute a multi-step trajectory to find the data it needs, backtracking and reacting to real-time information where necessary. The model is also able to browse over user uploaded files, plot and iterate on graphs using the python tool, embed both generated graphs and images from websites in its responses, and cite specific sentences or passages from its sources. As a result of this training, it reaches new highs on a number of public evaluations focused on real-world problems.

Part 8 of "A guide to implement ActivityPub in a static site (or any website)" by maho.dev recently dropped.

This part goes over how to add replies and comments to your static site.

Buc-ee's, the popular Texas-based travel center chain, is set to open its first Wisconsin location in Oak Creek, city officials announced.

Time to get my banana pudding fix.

This is something I've been dealing with on my site. I separate my long-form from short-form posts. However, I don't have a hard limit that clearly defines which is which. An initial reason for the separation was, I wanted to have a feed that was easy to scroll through. That was harder to do if I was rendering an entire long-form post as part of the feed. In the end though, most of my posts are all markdown files and their main distinction is how they are rendered.

As I work on my website redesign, I want to change how I think about posts. I want to separate the post content from the rendering. A post is a post regardless of content and length. Regardless of whether it's a response, image, video, note, or anything in between, I don't want to constrain the content of my post. I can choose to render posts differently depending on whether it's being viewed in a feed or individual post page. The content driving those different views though shouldn't change.

This is a cool walk-through of how to add semantic search to Emacs using local ML models.

With ONNX.el you can run any ML model inside Emacs, including ones for images, audios, etc. In case you are running embedding models, combine this with sem.el to perform semantic searches. Depending on your input modality, you might have to figure out preprocessing. For text, tokenizers.el should cover all of what you need in modern NLP for preprocessing, though you will have to do something on your own for anything non-text or multimodal.

By default sem runs a local all-MiniLM-L6-v2 model for text embeddings. This model is small and runs fast on CPU. Additionally, we load an O2 optimized variant which helps further. The vectors are stored in a lancedb database on file system which runs without a separate process. I was originally writing the vector db core myself to learn more of Rust, but then stopped doing that since lancedb gives all that I needed, out of the box.

...you needed to move anything you care about online to a space you control. Digital sovereignty is more important than ever.

💯💯💯💯💯

Best time was yesterday. Next best time is now.

Love this article.

Before the web was more commercialized, it was personal...A website was a way to represent yourself and connect to others, a space to tend and call your very own.

The imperfect, handmade character of websites has been sanded away in favor of efficiency, and social media platforms have risen in the stead of amateur sites. Many of us conform to these containers and retreat from expressing our authentic selves publicly...

If the idea of a more handcrafted internet resonates with you, the best way to be part of the movement is simply to make your own website. This may seem intimidating if we expect webpages to be a holistic reflection of ourselves, like a resume, portfolio, or blog. We limit the web as a medium when we expect that a website must fulfill a host of needs, or serve any function at all. A website doesn’t need to be anything but your own.

A more personal web becomes possible once we turn away from our preconceived notions of what a website must be. Let’s use our tools to continuously push at the boundaries of the web. Doing so will create a more creative, expansive environment that allows us to reclaim our agency and reinvigorate our collective understanding of software as an artisanal craft. Instead of concentrating on a few, monolithic sites, what if we visited many, more personalized sites? We should explore the far reaches of the web and invite our friends to join in: After all, we make the internet, and only we can build the web we want.

David Lynch, the filmmaker celebrated for his uniquely dark and dreamlike vision in such movies as “Blue Velvet” and “Mulholland Drive” and the TV series “Twin Peaks,” has died just days before his 79th birthday.

I didn't understand all of his work, but I appreciate his originality and overall aesthetic.

I'm going to tell future generations, this is how we built multi-agent systems back in my day.

Cyborg insects refer to hybrid robots that integrate living insects with miniature electronic controllers to enable robotic-like programmable control. These creatures exhibit advantages over conventional robots in adaption to complex terrain and sustained energy efficiency. Nevertheless, there is a lack of literature on the control of multi-cyborg systems. This research gap is due to the difficulty in coordinating the movements of a cyborg system under the presence of insects' inherent individual variability in their reactions to control input. Regarding this issue, we propose a swarm navigation algorithm and verify it under experiments. This research advances swarm robotics by integrating biological organisms with control theory to develop intelligent autonomous systems for real-world applications.

Shown off at CES 2025 is VLC's upcoming automatic subtitling feature, which is local in nature, running entirely on a user's machine without needing to connect to some distant server.

According to Jean-Baptiste Kempf, the creator and lead developer of VLC, this new feature is powered by open source AI models, which carry out subtitle generation tasks in two stages.

One aspect is automatic generation of subtitles from the video, and the other is translation of those subtitles into over 100 languages.

Unfortunately, there's no word on when this feature will be arriving on VLC.

Interesting interview and introduction to Calm Technology and the Calm Tech Institute.

When you choose to keep your notes in locally stored plain text files, you’re choosing to be your own landlord. Keeping notes in a proprietary app is the same as renting: the land you’re building on belongs to someone else, with no guarantee of continuing access.

This is so true for many other areas of our digital lives as well.

When we encounter something new and interesting, our brains release dopamine – the same neurotransmitter associated with reward and pleasure. This creates a simple pattern: the more we learn, the more we want to learn.

This drive to explore and understand isn’t just nice to have – it’s essential to who we are as humans.

There’s no single “right” way to be curious. Learning practical skills like coding or cooking, diving into topics like history or science, joining groups of people who share your interests… These are all great ways to inject more curiosity into your life.

Designing a year of curiosity

• Monthly: Design one tiny experiment at the beginning of each month This could be as simple as exploring a topic you know nothing about, trying a new hobby, or doing something that pushes you out of your comfort zone...
  
  
• Weekly: Every week, take 10 to 15 minutes to conduct a weekly review. What went well this week? What didn’t go as planned? What will you focus on next week? This could mean doubling down on what worked or tweaking something that didn’t...
  
  
• Daily: Create at least one moment of curiosity in your day, no matter how small. Experiment with a new recipe or tool, have one meaningful conversation, try a journaling prompt, or take a different route to work. Even one minute of curiosity a day can add up to a much richer life.

The unprecedented capabilities of foundation models have opened the door to agentic applications that were previously unimaginable. These new capabilities make it finally possible to develop autonomous, intelligent agents to act as our assistants, coworkers, and coaches. They can help us create a website, gather data, plan a trip, do market research, manage a customer account, automate data entry, prepare us for interviews, interview our candidates, negotiate a deal, etc. The possibilities seem endless, and the potential economic value of these agents is enormous.

This section will start with an overview of agents and then continue with two aspects that determine the capabilities of an agent: tools and planning. Agents, with their new modes of operations, have new modes of failure. This section will end with a discussion on how to evaluate agents to catch these failures.

Humans are fantastic at messy pattern recognition tasks. However, they often rely on tools - like books, Google Search, or a calculator - to supplement their prior knowledge before arriving at a conclusion. Just like humans, Generative AI models can be trained to use tools to access real-time information or suggest a real-world action. For example, a model can leverage a database retrieval tool to access specific information, like a customer's purchase history, so it can generate tailored shopping recommendations. Alternatively, based on a user's query, a model can make various API calls to send an email response to a colleague or complete a financial transaction on your behalf. To do so, the model must not only have access to a set of external tools, it needs the ability to plan and execute any task in a self- directed fashion. This combination of reasoning, logic, and access to external information that are all connected to a Generative AI model invokes the concept of an agent, or a program that extends beyond the standalone capabilities of a Generative AI model. This whitepaper dives into all these and associated aspects in more detail.

On January 1, 2025, we celebrate published works from 1929 and published sound recordings from 1924 entering the public domain! The passage of these works into the public domain celebrates our shared cultural heritage. The ability to breathe new life into long forgotten works, remix the most popular and enduring works of the time, and to better circulate the oddities we find in thrift stores, attics, and on random pockets of the internet are now freely available for us all.

While not at the same blockbuster level as 2024 with Steamboat Willie’s passage into the public domain, works from 1929 still inhabit strong cultural significance today. The works of 1929 continue to capture the Lost Generation’s voice, the rise of sound film, and the emerging modern moment of the 1920s.

In 2024, developers leaned into complexity with multi-step agents, sharpened efficiency by doing more with fewer LLM calls, and added quality checks to their apps using methods of feedback and evaluation.

MarkItDown is a utility for converting various files to Markdown (e.g., for indexing, text analysis, etc). It supports:

• PDF
• PowerPoint
• Word
• Excel
• Images (EXIF metadata and OCR)
• Audio (EXIF metadata and speech transcription)
• HTML
• Text-based formats (CSV, JSON, XML)
• ZIP files (iterates over contents)

The "Agentic Fediverse" is the idea of a new kind of network federation, a complementary iteration on the concept of federation currently established with ActivityPub. It's an experiment and something that we in Muni Town are still trying to define more concretely.

Tenets

We are still working on defining the core tenets of the agentic fediverse, but here is what we have so far.

An agentic fediverse is...

1. A fediverse of agents; agent-centric, as opposed to server-centric.
2. Local-first; solve local problems for local people.
3. Accessible by default; inaccessibility is a bug.
4. Systematically consensual; architected on the basis of informed consent.

We're A New Social, a new non-profit organization focused on building cross-protocol tools and services for the open social web.

Our mission is to liberate people's networks from their platforms, enabling The Last Network Effect and leveling the playing field across the open social web.

The first project we'll take on to accomplish this mission is Bridgy Fed, a service that enables users of ActivityPub-based platforms like Mastodon, ATProto-based platforms like Bluesky, and websites to interact and engage across ecosystems.

Interesting post. Whether building your feeds or plugging into existing platforms like Bluesky, this framework could serve as a good starting point.

Today, we’re abnormally jazzed to announce that we’re open-sourcing the custom framework we built to power your dashboard on Tumblr. We call it StreamBuilder, and we’ve been using it for many years.

StreamBuilder has a lot going on. The primary architecture centers around “streams” of content: whether posts from a blog, a list of blogs you’re following, posts using a specific tag, or posts relating to a search. These are separate kinds of streams, which can be mixed together, filtered based on certain criteria, ranked for relevancy or engagement likelihood, and more.

So, what’s included in the box?

• The full framework library of code that we use today, on Tumblr, to power almost every feed of content you see on the platform.
• A YAML syntax for composing streams of content, and how to filter, inject, and rank them.
• Abstractions for programmatically composing, filtering, ranking, injecting, and debugging streams.
• Abstractions for composing streams together—such as with carousels, for streams-within-streams.
• An abstraction for cursor-based pagination for complex stream templates.
• Unit tests covering the public interface for the library and most of the underlying code.

GitHub Repo

Openverse is a tool that allows openly licensed and public domain works to be discovered and used by everyone.

Openverse searches across more than 800 million images and audio tracks from open APIs and the Common Crawl dataset. We aggregate works from multiple public repositories, and facilitate reuse through features like one-click attribution.

Openverse is the successor to CC Search which was launched by Creative Commons in 2019

Cool retrospective and announcements from this WordPress' State of the Word 2024.

Focused mode and templates look interesting.

Tumblr is introducing a new Community feature — in-app groups organized by topic or interest.

Communities are similar to subreddits or Facebook groups and had previously been in beta.

Given recent announcements like Tumblr replatforming on top of WordPress and WordPress adding support for ActivityPub, I could see communities becoming not only an alternative to Reddit, but also Lemmy in the Fediverse.

Great post from jordanmatthiesen.me.

Recently on a trip for a tech conference I created a YouTube chat app using .NET and AI. This is part of my exploration into creating a larger app for chatting about .NET AI development (leveraging docs, presentations, and sample code my team has been working on at Microsoft).

Our video generation model is rolling out at sora.com⁠.

Users can generate videos up to 1080p resolution, up to 20 sec long, and in widescreen, vertical or square aspect ratios. You can bring your own assets to extend, remix, and blend, or generate entirely new content from text.

The version of Sora we are deploying has many limitations. It often generates unrealistic physics and struggles with complex actions over long durations.

We’re introducing our video generation technology now to give society time to explore its possibilities and co-develop norms and safeguards that ensure it’s used responsibly as the field advances.

...my blog has become a central hub for almost everything I write online, including nearly all my social media posts, links to my published work on other sites, and the occasional freestanding blog post. While this approach has a few trade-offs, I feel better about posting in the first place knowing that I’m doing it on my own terms.

There’s also just something deeply satisfying about seeing a running feed of my own writing, on my own website, with my own design and presentation. A social network alone can’t replicate that, but the POSSE approach offers the best of both worlds.

I've enjoyed seeing over the past few weeks, as people seek new online platforms, the personal website is being rediscovered.

😂 😂 😂

I don't think I've ever heard James Franklin called a gambler.

The game against Minnesota was too close for comfort, but a win is a win. In many ways, thanks to Jimmy, The Gambler.

I've been following along with this amazing series from Maho on implementing ActivityPub on static websites.

As I think about what I want out of my website and the way I engage in the Fediverse, there's a lot of overlap.

Today, what I mainly use my Mastodon instance for is:

1. Having a Fediverse presence on a self-hosted Mastodon instance.
2. Cross-posting posts on by website using the POSSE pattern.

Although I have learned a ton from self-hosting my own Mastodon instance, neither of the points listed above require me self-hosting or even having an account on someone else's instance.

Yesterday, I took the first step in linking my Fediverse presence to my domain. That fulfills my first requirement.

I rarely post original content on Mastodon and for consumption, I already subscribe to accounts and tags via RSS. Therefore, the second requirement is one that can naturally occur without having to use a Mastodon instance as an intermediary. I can just post on my website and because my posts can show up in my outbox, there's no need to POSSE. I don't want my presence to be limited to Mastodon though, but since I plan on supporting media, reviews, and other types of posts, I expect my posts to be accessible across other platforms on the Fediverse like Pixelfed, Bookwyrm, and many others.

Since my website and website features are built using .NET, Maho's guide simplifies my implementation. Because I'm not using any of the existing static site generators and rolled my own, that means that there's going to be some effort and customizations required on my end. It would no different from my custom Webmentions implementation though. My hope is that I will save time I often spend maintaining the server as well as money since I no longer need to rent a server to host my instance. At the same time, I get to learn and contribute to building a more open, decentralized, and personal web.

I saw a reference to this paper yesterday and added it to my Read-It-Later list.

Here's a link to the original paper: https://philpapers.org/archive/AYLITA.pdf

Chances are, I wasnt going to get to it until way later with the backlog of other things I want to read later.

Lucky for me, that was the subject of the latest Deep Questions episode. Cal does a nice job breaking down the main arguments of the paper.

The main takeaway for me was, technology is a great tool. However, when it robs you of your rational decision making abilities and pushes you to compulsive behaviors that move you away from the habits and practices that fuel your growth, it's a problem. For example, lets say you want to build a habit of reading or working out. Unfortunately, when you , time, attention, and energy that would've otherwise been spent on those habits is now diverted towards the distraction. That is irrational and goes against your desire to grow, yet making the rational choice is where most of us struggle. Although the paper focuses on our digital lives, I can see other ways similar arguments can be applied.

I've also included the audio version as well: https://www.buzzsprout.com/1121972/episodes/16124495-ep-327-would-kant-use-tiktok

This video showed up in my feed yesterday. LuvstarKei makes some great points for building your own space on the web.

With the most recent Twitter / X exodus, it's great to see folks moving to places built on more open protocols like AT (Bluesky) or ActivityPub (Fediverse).

That said, those platforms are not your own, even in cases where you self-host. If tomorrow Threads, Bluesky, or Mastodon stop hosting your instance or stop development altogether, what happens to the content and your community? In the case of the Fediverse, many projects are being built in the open and there are other providers you could switch to. The switching costs though may not make it worth bothering to export and keep your content as is usually the case. That's not to say with a personal website you might not have similar issues (i.e. switching from WordPress to Ghost). However, HTML is HTML and you can rehost that anywhere on the internet.

Everyone is looking for different things from the social platforms, and that's not to say they don't have their value. However, from an ownership, permanence, and creative freedom perspective, nothing beats having your own website.

The recordings are out for .NET Conf 2024 in case you missed any of the sessions.

Here are the recordings from the sessions I participated in.

• Building AI Applications From Scratch: A Hands-On Guide for .NET Developers
• Building the Foundation: AI Fundamentals in .NET

Also, there's a Premier Bonus playlist which has a ton of amazing bonus content.

Love this new "making of" video. I was surprised to learn Thundercat is on the new album. I didn't catch which track it's on, but will be listening for it.

Day 1 of .NET Conf was a lot of fun. Tons of great sessions. Here's the recording of all the sessions.

If you're interested in checking out my session with Jeremy, Building AI Applications From Scratch, here's a link to it.

Tomorrow, I'll also get a chance to share the stage with Tarek and Tanner to talk AI Fundamentals.

So cool to see you posting more on your own platform.

Introducing DRIFT Search: Combining global and local search methods to improve quality and efficiency

DRIFT search (Dynamic Reasoning and Inference with Flexible Traversal)...builds upon Microsoft’s GraphRAG technique, combining characteristics of both global and local search to generate detailed responses in a method that balances computational costs with quality outcomes.

DRIFT Search: A step-by-step process

1. Primer: When a user submits a query, DRIFT compares it to the top K most semantically relevant community reports. This generates an initial answer along with several follow-up questions, which act as a lighter version of global search. To do this, we expand the query using Hypothetical Document Embeddings (HyDE), to increase sensitivity (recall), embed the query, look up the query against all community reports, select the top K and then use the top K to try to answer the query. The aim is to leverage high-level abstractions to guide further exploration.
2. Follow-Up: With the primer in place, DRIFT executes each follow-up using a local search variant. This yields additional intermediate answers and follow-up questions, creating a loop of refinement that continues until the search engine meets its termination criteria, which is currently configured for two iterations (further research will investigate reward functions to guide terminations). This phase represents a globally informed query refinement. Using global data structures, DRIFT navigates toward specific, relevant information within the knowledge graph even when the initial query diverges from the indexing persona. This follow-up process enables DRIFT to adjust its approach based on emerging information.
3. Output Hierarchy: The final output is a hierarchy of questions and answers ranked on their relevance to the original query. This hierarchical structure can be customized to fit specific user needs. During benchmark testing, a naive map-reduce approach aggregated all intermediate answers, with each answer weighted equally.

METAREFLECTION: Learning Instructions for Language Agents using Past Reflections

Language agents are AI systems that can understand, reason and respond in natural language to complete various tasks. While the latest LLMs are capable enough to power reasonably good language agents, the closed-API model makes it hard to improve them when they perform sub-optimally. Recent studies have explored using techniques like self-reflection and prompt optimization to improve performance. Unfortunately, self-reflection can be used only during the agent’s current run, while contemporary prompt optimization techniques are designed and tested to work on simple single-step agents.

In a recent paper: METAREFLECTION: Learning Instructions for Language Agents using Past Reflections, researchers from Microsoft introduce a novel offline reinforcement learning technique that enhances the performance of language agents by augmenting a semantic memory based on experiential learnings from past trials. They demonstrate the efficacy of METAREFLECTION across multiple domains, including complex logical reasoning, biomedical semantic similarity, open world question answering, and vulnerability threat detection, in Infrastructure-as-Code, spanning different agent designs. METAREFLECTION boosts language agents’ performance by 4% to 16.82% over the baseline agent implementations and performs on par with existing state-of-the-art prompt optimization techniques while requiring fewer LLM calls.

Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping

Generative AI applications rely on large, foundation models, particularly LLMs. LLMs often have tens to hundreds of billions of parameters, making them too large for a single graphics processing unit (GPU) to handle in terms of both memory and computation. Because of their size, training these models requires distributing the workload across hundreds or even thousands of GPUs. This can lead to significant communication overhead, a challenge that arises when data needs to be shared between different GPUs.

In a recent paper: Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping, researchers from Microsoft introduce a system designed to enhance the efficiency of LLM training by reducing the time lost to communication between GPUs.

Domino breaks down data dependencies in a single batch of training into smaller, independent pieces. These smaller pieces are processed in parallel, and communication between GPUs happens simultaneously with computation, minimizing delays.

Test results comparing Domino to Megatron-LM show that Domino speeds up the training process by up to 1.3x on Nvidia DGX-H100 GPUs.

OmniParser for pure vision-based GUI agent

Large vision-language models (VLMs) such as GPT-4V and GPT-4o show promise in driving intelligent agent systems that operate within user interfaces (UI). However, VLMs’ full potential remains underexplored in real-world applications, particularly when it comes to acting as general agents across diverse operating systems and applications with only vision input. One limiting factor is the absence of a robust technique for screen parsing which is capable of 1) reliably identifying interactable icons within the user interface, and 2) understanding the semantics of various elements in a screenshot and accurately associating the intended action with the corresponding region on the screen.

In a recent article: OmniParser for pure vision-based GUI agent, researchers from Microsoft present a compact screen parsing module that can convert UI screenshots into structured elements. OmniParser can be used with a variety of models to create agents capable of taking actions on UIs. When used with GPT-4V, OmniParser significantly improves the agent capability to generate precisely grounded actions for interface regions.

OmniParser with GPT-4V agent achieved the best performance on the recently released  WindowsAgentArena (opens in new tab) benchmark.

Recent research arXiv:2410.15027 has explored the use of diffusion transformers (DiTs) for task-agnostic image generation by simply concatenating attention tokens across images. However, despite substantial computational resources, the fidelity of the generated images remains suboptimal. In this study, we reevaluate and streamline this framework by hypothesizing that text-to-image DiTs inherently possess in-context generation capabilities, requiring only minimal tuning to activate them. Through diverse task experiments, we qualitatively demonstrate that existing text-to-image DiTs can effectively perform in-context generation without any tuning. Building on this insight, we propose a remarkably simple pipeline to leverage the in-context abilities of DiTs: (1) concatenate images instead of tokens, (2) perform joint captioning of multiple images, and (3) apply task-specific LoRA tuning using small datasets (e.g., 20∼100 samples) instead of full-parameter tuning with large datasets. We name our models In-Context LoRA (IC-LoRA). This approach requires no modifications to the original DiT models, only changes to the training data. Remarkably, our pipeline generates high-fidelity image sets that better adhere to prompts. While task-specific in terms of tuning data, our framework remains task-agnostic in architecture and pipeline, offering a powerful tool for the community and providing valuable insights for further research on product-level task-agnostic generation systems.

Repo: https://github.com/ali-vilab/In-Context-LoRA

LLMs are typically trained to answer user questions or follow instructions similarly to how human experts respond. However, in the standard alignment framework they lack the basic ability of explicit thinking before answering. Thinking is important for complex questions that require reasoning and planning -- but can be applied to any task. We propose a training method for equipping existing LLMs with such thinking abilities for general instruction following without use of additional human data. We achieve this by an iterative search and optimization procedure that explores the space of possible thought generations, allowing the model to learn how to think without direct supervision. For each instruction, the thought candidates are scored using a judge model to evaluate their responses only, and then optimized via preference optimization. We show that this procedure leads to superior performance on AlpacaEval and Arena-Hard, and shows gains from thinking on non-reasoning categories such as marketing, health and general knowledge, in addition to more traditional reasoning & problem-solving tasks.

The “have to” narrative positions us as repositories of instructions made elsewhere, as if we were just programs following the code we’ve been given...

The “choose to” narrative has no illusions about our power and recognizes that we are small players in a bigger, and certainly unjust, world. But we are not machines. And maybe we don’t like the choices available to us, maybe we wish there were others within reach. But once we accept that there are choices to make, we may notice where we have some room to maneuver, some space to play with, some opportunity or avenue or loophole we can exploit.

Refusing your own agency time and again is like disconnecting from a power source—the energy is still there, latent and ready, but the plug dangles inches from the outlet.

...accepting that you have some agency might hurt: you bump up against the systems that constrain your choices; you see more clearly how other people’s choices limit (or expand) your own. But it keeps you connected to that source, that font of energy that is yours and no one else’s. It keeps you hooked up to who you are, and to what you want.

Interesting excerpts from Mandy Brown's "Haves and Choices" article to think about.

This is cool! Already looking at how I can repurpose this sample for a project I have in mind.

With Terrifier 3 now playing in theaters and just passing $50 million worldwide, Art the Clown’s domination of the Halloween season continues with a takeover of the Nasdaq MarketSite in Times Square!

...Art the Clown and star Lauren LaVera will ring the Nasdaq closing bell!

This is so cool! Hopefully that's all he does.

He's back! A day after Tyler too.

I already loved Garmonbozia but Ajhussi is my favorite track.

Although on this album you hear some of the early FlyLo sound, it still feels new and fresh.

I did a first pass earlier today and I generally enjoyed this new album from Tyler.

Like many previous albums, it's highly introspective. I loved the instrumentals as well.

Some first impressions:

• Track 10 is not a double track as it's generally been the case of his previous albums.
• "Like Him" adds a new contrasting dimension to Tyler's relationship with his father.

I'll need some more time to digest it and read through the lyrics but overall, great album.

Sad news from the Tetons this week. Grizzly 399 passed away after being struck by a vehicle.

Grizzly No. 399 died Tuesday night on a highway in Snake River Canyon south of Jackson...

At 28 years old, No. 399 was the oldest known reproducing female grizzly in the Yellowstone ecosystem. Each spring, wildlife enthusiasts eagerly awaited her emergence from her den to see how many cubs she had birthed over the winter — then quickly shared the news online.

The grizzly lived through a time of strife over her species in the region...

I think this is Kamasi's first time on Tiny Desk. Great performance. I didn't realize Brandon Coleman was on keys. The encore, "Asha The First", was really good.

I'm ready for this new album. It reminds me of Cherry Bomb in some ways. Love the instrumentals.

2024 just keeps delivering. Most of the artists I follow have dropped an album this year. Now I'm just waiting on FlyLo. Hopefully he releases his album by end of year.

I don't know what to expect from Chromakopia, but I can't wait after watching this trailer.

Yes, you should!

...the conduit through which I experience Molly's excellent work is totally enshittification-proof, and the more I use it, the easier it is for everyone to be less enshittified.

This conduit is anti-lock-in, it works for nearly the whole internet. It is surveillance-resistant, far more accessible than the web or any mobile app interface. It is my secret super-power.

It's RSS.

...But RSS isn't just good for the news! It's good for everything.

Your RSS reader doesn't (necessarily) have an algorithm. By default, you'll get everything as it appears, in reverse-chronological order.

Now, you sign up to so many feeds that you're feeling overwhelmed and you want an algorithm to prioritize posts – or recommend content. Lots of RSS readers have some kind of algorithm and recommendation system...

But you control the algorithm, you control the recommendations. And if a new RSS reader pops up with an algorithm you're dying to try, you can export all the feeds you follow with a single click, which will generate an OPML file. Then, with one click, you can import that OPML file into any other RSS reader in existence and all your feeds will be seamlessly migrated there. You can delete your old account, or you can even use different readers for different purposes.

RSS basically works like social media should work. Using RSS is a chance to visit a utopian future in which the platforms have no power, and all power is vested in publishers, who get to decide what to publish, and in readers, who have total control over what they read and how, without leaking any personal information through the simple act of reading.

And here's the best part: every time you use RSS, you bring that world closer into being!

Unlike those largely useless, performative boycotts of widely used platforms, switching to RSS doesn't require that you give anything up. Not only does switching to RSS let you continue to follow all the newsletters, webpages and social media accounts you're following now, it makes doing so better: more private, more accessible, and less enshittified.

Using RSS to follow the stuff that matters to you will have an immediate, profoundly beneficial impact on your own digital life – and it will appreciably, irreversibly nudge the whole internet towards a better state.

Great article by Cory Doctorow.

I've written at length on my website as to why I enjoy RSS. My use goes back to the days of Google Reader. Like many, once that was disbanded, my consumption pretty much shifted to social media for a few years. After growing dissatisfaction with the limitations and lack of control over my feed, I started my journey back to RSS readers eventually landing on NewsBlur and elfeed as my readers of choice.

On this website, RSS plays an important role in a few areas.

RSS is how I:

• Enable discovery of my own feeds
• Enable visitors to subscribe and get the latest content
• Syndicate posts to Mastodon.
• Enable feed discovery of websites I subscribe to through my:
  • Blogroll
  • Podroll (podcasts)
  • YouTubeRoll?
  • ForumRoll?

So how can you get started?

• Learn about feeds.
• Check out the many getting started guides.
• Browse through blogrolls and feed catalogs
• Explore various feed readers to find the one that works best for you.

Videos, Slides, Presentations, and assets from DEFCON.

I haven't watched the entire talk, but here are some excerpts from the transcript that resonated with me.

For a lot of people, the web feels worse than it used to. Much of our time is spent on a handful of giant social networks that do everything they can to keep people in their apps for as long as possible, even if it has detrimental effects on the people who are using them, or on the social networks themselves.

Websites outside of these handful of social networks are harder and harder to even find, and partially because of that, they have a harder and harder time sustaining themselves.

What is left for those of us who saw the web as an infinite canvas, a tool to reach those who we never could have dreamed of reaching before, a medium that could stretch the limits of what was even possible in an analog world?

...having said all this, would it surprise you to hear that now more than ever, I feel that same burning feeling of excitement around what’s possible?

That’s because what really sucks about the web these days, what has us feeling despair and anger, has everything to do with the industry that has formed around the web, but not the web itself. The web is still just a substrate on which anything can be built. Most importantly, the web is the people who use it, not the companies that have established themselves around it.

And the widespread disillusionment that we’re seeing may actually be a good thing. More people than ever have realized that the utopian dreams of a web that could only bring about positive and wonderful things might have been misguided.

With this knowledge comes power. The power to shape the web that we want to see, while fighting against the one that we don’t.

My experience in fighting this fight has helped to convince me of just how much power we, everyday normal people, have—even when we’re staring down massive platforms and billion-dollar companies.

I ended up doing what I enjoy doing—building something cool, mostly for the sake of building it, and writing down what I saw.

...this simple act of building something interesting and somewhat different had an impact that I could not have anticipated. It turned out that people were starving for it.

And even though I was a nobody, it had a major impact.

The lesson was clear. The tech industry, even with its billions of dollars, is not an indomitable force. Though they can and will ignore what people want from them, they cannot control what those people think. And the tide can turn against them. And it doesn’t always take a job at The New York Times or a huge pre-established platform to become one of the voices speaking up, helping to turn that tide. Sometimes you just have to make something cool. And using the very same technology that enabled crypto guys to sell their scam tokens, or the boosterist journalists to publish obsequious descriptions of companies that were really selling vaporware, or the crypto community to spam social media with promotions for their NFTs, I was able to do something a little different to push back against that very same phenomenon.

What they don’t seem to realize is that in doing so, by reducing the web only to the types of expression that can happen within their cramped boxes—where you can’t write more than 280 characters, or you can’t publish your cool JavaScript-based art project, or you can’t say the things that you want to say without getting de-boosted by the engagement maximization machine, or you can’t read what your friends are posting without the platform interjecting offensive troll posts or soulless AI-generated meme images—they’re creating a thirst for everything outside of those boxes.

A thirst for what the web really is—a medium, a conduit, a tool that is used by readers and artists and creators and explorers, not a gatekeeper that seems to be in an ever more adversarial relationship with everyone who uses it.

The platforms I talk about are deeply entrenched to the point where many people barely use the web outside of them.

But the platforms do not exist without the people, and there are a lot more of us than there are of them.

...it has never been easier to do cool things on the web. What used to be expensive and require a great deal of technical expertise is now becoming more and more accessible, both financially and technologically, by the day. More people than ever have access to the web, both in terms of access to devices, but also in terms of access to internet connectivity and software. What used to be the realm of the nerds and those with the financial wherewithal to purchase expensive home computers, connectivity, and software packages, is now home to people from all walks of life, who bring new ideas, perspectives, and experiences that we often forget were in short supply during what some of us think of as the “good old days” of the internet.

And so, we—all of us—can build the projects we want to see, write the software that the big platforms won’t, and create the services that people need. We can wire everything together, whether the platforms like it or not, to tear down the walls that they put up. We can modify the software, reverse engineer their systems, and wrestle back control of how we experience the web, even through these very platforms. We can share information and help people understand what is happening around them. We can teach others to build the things they want to build, and share with one another the important and meaningful work that is being done.

We can build the web that we want to see, and we can return to that place where the web is a place of wonder, where all of us feel that same burning feeling of excitement as we push the web back towards the wonderful, beautiful, joyful place it ought to be.

This is such a beautiful album. It seems I'm not the only one that thinks so since it's rated as the #1 album of all time by Rolling Stone. The powerful messages in the lyrics still resonate to this day and the composition is such a perfect match for Marvin Gaye's voice. 50 years later it feels like this album has transcended space and time.

Over the past few days, I've listened to it from beginning to end at least 3 times.

Of course there's the classics like What's Going On and Mercy Mercy Me that many people know. However, the ones that have quickly become my favorites are:

Save The Children

God Is Love

Simply amazing.

I love the visuals on this video.

Since Automattic acquired Tumblr we’ve made it more efficient, grown its revenue, and worked to improve the platform. But there’s one part of the plan that we haven’t yet started, which is to run Tumblr on WordPress. I’m pleased to say we’re kicking off that project now!

A little late, but I can't express how much I love this.

We love Tumblr’s streamlined posting experience and its current product direction. We’re not changing that. We’re talking about running Tumblr’s backend on WordPress. You won’t even notice a difference from the outside.

Although it sounds outside the scope of the original plan, it would be amazing if the shared backend as well as the Tumblr publishing front-end made their way into the open-source version of WordPress.

That would open the door to self-hosted personal websites that also make it significantly easier to post content in smaller chunks like a microblog.

Pair that with ActivityPub integrations and you now have a connected web of individuals where the first place they post content to is their own website and for discovery / broader reach can also federate and post to other platforms that also support protocols like ActivityPub.

Can't wait to see how this project develops.

Do you yearn for the days when people owned their corner of the internet and expressed themselves in wild and wacky ways? Do you want to see an internet focused on creativity, art, and ideas instead of debating and dividing? Do you think content and data should be owned by authors and artists, instead of getting locked behind the closed platform of a mega-corporation? Do you want to build an internet where anyone with a story can tell it, and anyone with a product can sell it, regardless of income, gender, politics, language, or where they live in the world?

The answer to all of those is YES.

I'm excited to be at DEVintersection again this year where I'll get a chance to meet old and new friends.

I have a few sessions where I'll be talking about some of my favorite things, .NET and AI.

• Introduction to AI in .NET
• Building AI Applications from Scratch: A Hands-On Guide for .NET Developers
• Ask The Experts - Starting out on your .NET & AI explorations

Other sessions I recommend as well:

• Navigating the World of AI Models in .NET: From Local Development to the Cloud
• KEYNOTE - Practical Real-World AI for Developers
• Making Web Applications Intelligent and Performant with Redis
• Building AI Copilots: Integrate Semantic Kernel, Azure OpenAI, and Azure Cosmos DB with .NET Aspire
• Introduction to .NET Aspire

There's so many more, but in the interest of not listing them all out, check out the schedule.

See you there!

I came across this gem over the weekend. Mikaela Davis on the harp just takes Bird Song and Ripple to the next level.

At AI Infra @ Scale 2024, Meta engineers discussed every step of how we built and brought Llama 3 to life, from data and training to inference.

We present a music recommendation ranking system that uses Transformer models to better understand the sequential nature of user actions based on the current user context.

As above, so below.

Some new Chicano Batman was a pleasant surprise this morning. Even better, an instrumental track.

You can hear some of their older sound in this track, which I really enjoyed.

I've had this song on repeat sinceFriday. Also, I didn't know the name of the song is a Twin Peaks reference. Given his recent focus as a composer, I hope this means there's a bigger Twin Peaks project or collaboration in the works.

Here's something I didn't know I needed in my life. Perfect way to start a Saturday morning.

Last night Organic Maps was removed from the Play Store without any warnings or additional details due to "not meeting the requirements for the Family Program". Compared to Google Maps and other maps apps rated for 3+ age, there are no ads or in-app purchases in Organic Maps. We have asked for an appeal.

The app is still available on F-Droid. Much better place for getting apps in my opinion.

Good way to start a Friday. New drop from FlyLo. I like it.

Transformers have revolutionized machine learning, yet their inner workings remain opaque to many. We present Transformer Explainer, an interactive visualization tool designed for non-experts to learn about Transformers through the GPT-2 model. Our tool helps users understand complex Transformer concepts by integrating a model overview and enabling smooth transitions across abstraction levels of mathematical operations and model structures. It runs a live GPT-2 instance locally in the user's browser, empowering users to experiment with their own input and observe in real-time how the internal components and parameters of the Transformer work together to predict the next tokens. Our tool requires no installation or special hardware, broadening the public's education access to modern generative AI techniques. Our open-sourced tool is available at this https URL. A video demo is available at this https URL.

• Tool
• Video

Extraction and interpretation of intricate information from unstructured text data arising in financial applications, such as earnings call transcripts, present substantial challenges to large language models (LLMs) even using the current best practices to use Retrieval Augmented Generation (RAG) (referred to as VectorRAG techniques which utilize vector databases for information retrieval) due to challenges such as domain specific terminology and complex formats of the documents. We introduce a novel approach based on a combination, called HybridRAG, of the Knowledge Graphs (KGs) based RAG techniques (called GraphRAG) and VectorRAG techniques to enhance question-answer (Q&A) systems for information extraction from financial documents that is shown to be capable of generating accurate and contextually relevant answers. Using experiments on a set of financial earning call transcripts documents which come in the form of Q&A format, and hence provide a natural set of pairs of ground-truth Q&As, we show that HybridRAG which retrieves context from both vector database and KG outperforms both traditional VectorRAG and GraphRAG individually when evaluated at both the retrieval and generation stages in terms of retrieval accuracy and answer generation. The proposed technique has applications beyond the financial domain

Given my recent experience unsubscribing from content, there's definitely room for improvement.

GPT-4o is an autoregressive omni model, which accepts as input any combination of text, audio, image, and video and generates any combination of text, audio, and image outputs. It\u2019s trained end-to-end across text, vision, and audio, meaning that all inputs and outputs are processed by the same neural network.

GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time(opens in a new window)2 in a conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models.

In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House3, we are sharing the GPT-4o System Card, which includes our Preparedness Framework(opens in a new window)5 evaluations. In this System Card, we provide a detailed look at GPT-4o\u2019s capabilities, limitations, and safety evaluations across multiple categories, with a focus on speech-to-speech (voice)A while also evaluating text and image capabilities, and the measures we\u2019ve taken to enhance safety and alignment. We also include third party assessments on general autonomous capabilities, as well as discussion of potential societal impacts of GPT-4o text and vision capabilities.

Nice! I remember reading Protocols, Not Platforms many years ago and felt inspired to seek out a better web.

JUST $100 million. That's pocket change 🙃

Red Ventures paid $500 million for the tech property once valued at $1.8 billion.

Given previous valuations though, I see why the headline was phrased that way.

A Blog Directory is a cool project from JC (Probably) and Lou Plummer. Check it out!

I no longer have an iPhone, but whenever anyone asks for a podcast app recommendation, Overcast is the first one I mention. Congrats on 10 years. Here's to many more.

I'm all for compensation. Remind me again, how much are the communities that create the content and make Reddit what it is getting out of these licensing deals?

This is a clever use of .well-known and OPML.

Definitely something I want to experiment with and implement on my site even if it's not widely adopted.

Large context window is a desirable feature in large language models (LLMs). However, due to high fine-tuning costs, scarcity of long texts, and catastrophic values introduced by new token positions, current extended context windows are limited to around 128k tokens. This paper introduces LongRoPE that, for the first time, extends the context window of pre-trained LLMs to an impressive 2048k tokens, with up to only 1k fine-tuning steps at within 256k training lengths, while maintaining performance at the original short context window. This is achieved by three key innovations: (i) we identify and exploit two forms of non-uniformities in positional interpolation through an efficient search, providing a better initialization for fine-tuning and enabling an 8x extension in non-fine-tuning scenarios; (ii) we introduce a progressive extension strategy that first fine-tunes a 256k length LLM and then conducts a second positional interpolation on the fine-tuned extended LLM to achieve a 2048k context window; (iii) we readjust LongRoPE on 8k length to recover the short context window performance. Extensive experiments on LLaMA2 and Mistral across various tasks demonstrate the effectiveness of our method. Models extended via LongRoPE retain the original architecture with minor modifications to the positional embedding, and can reuse most pre-existing optimizations.

GitHub Repo

I've been enjoying reading through this Tensors series.

If you're interested, here's also the link to part 1.

Takeaways

• Following up on the success of the Meta Segment Anything Model (SAM) for images, we're releasing SAM 2, a unified model for real-time promptable object segmentation in images and videos that achieves state-of-the-art performance.
• In keeping with our approach to open science, we're sharing the code and model weights with a permissive Apache 2.0 license.
• We're also sharing the SA-V dataset, which includes approximately 51,000 real-world videos and more than 600,000 masklets (spatio-temporal masks).
• SAM 2 can segment any object in any video or image - even for objects and visual domains it has not seen previously, enabling a diverse range of use cases without custom adaptation.
• SAM 2 has many potential real-world applications. For example, the outputs of SAM 2 can be used with a generative video model to create new video effects and unlock new creative applications. SAM 2 could also aid in faster annotation tools for visual data to build better computer vision systems.

Saving this guide for future reference as I set up my elfeed / Nyxt capture workflows for the website.

Additional articles that might be helpful.

Org capture in Nyxt: Taking Notes While Browsing

In my experience, the supermajority of engagement on viral AI Facebook pages is just as artificially-generated as the content they publish.

Whether it's a child transforming into a water bottle cyborg, a three-armed flight attendant rescuing Tiger Jesus from a muddy plane crash, or a hybrid human-monkey baby being stung to death by giant hornets, all tend to have copy+pasted captions, reactions & comments which usually make no sense in the observed context.

I've noticed similar patterns on YouTube. Sometime the comments include timestamp links which makes them seem more credible, but upon further inspection, it's all bot activity.

Testing org-capture template generated response file

The emerging golden age of home-cooked software, barefoot developers, and why the local-first community should help build it

This is a talk I presented Local-first Conference in Berlin, May 2024. It's specifically directed at the local-first community, but its relevant to anyone involved in building software.

For the last ~year I've been keeping a close eye on how language models capabilities meaningfully change the speed, ease, and accessibility of software development. The slightly bold theory I put forward in this talk is that we're on a verge of a golden age of local, home-cooked software and a new kind of developer – what I've called the barefoot developer.

Large Language Models (LLMs) have demonstrated remarkable capabilities across a variety of software engineering and coding tasks. However, their application in the domain of code and compiler optimization remains underexplored. Training LLMs is resource-intensive, requiring substantial GPU hours and extensive data collection, which can be prohibitive. To address this gap, we introduce Meta Large Language Model Compiler (LLM Compiler), a suite of robust, openly available, pre-trained models specifically designed for code optimization tasks. Built on the foundation of Code Llama, LLM Compiler enhances the understanding of compiler intermediate representations (IRs), assembly language, and optimization techniques. The model has been trained on a vast corpus of 546 billion tokens of LLVM-IR and assembly code and has undergone instruction fine-tuning to interpret compiler behavior. LLM Compiler is released under a bespoke commercial license to allow wide reuse and is available in two sizes: 7 billion and 13 billion parameters. We also present fine-tuned versions of the model, demonstrating its enhanced capabilities in optimizing code size and disassembling from x86_64 and ARM assembly back into LLVM-IR. These achieve 77% of the optimising potential of an autotuning search, and 45% disassembly round trip (14% exact match). This release aims to provide a scalable, cost-effective foundation for further research and development in compiler optimization by both academic researchers and industry practitioners.

• HuggingFace Collection
• Paper

Nice job by Cal debunking misconceptions about AI model capabilities. The segment highlights a few points I cover in my unpublished NoLM - Not Only Language Models blog post. Specifically the fact that Language Models on their own can't do much and need to be connected to data sources and other systems. Complex AI systems will be built with more specialized roles and leverage various components for their planning and execution. In the end though, models will require integration into existing systems. Those integrations need to be done by people, meaning humans are still in control of the AI-assisted system capabilities.

Later in the podcast, Cal takes a question about distributed webs of trust. I agree with Cal's point of using existing open standards like RSS for content consumption. It's the reason you often hear the phrase, "or wherever you get your podcasts". Assuming you have a program that can read an RSS feed, you can follow all types of content. On the topic of discovery, Cal makes the suggestion of using distributed webs of trust. Using domain names and linking as ways of discovering content. While blogrolls were not directly called out, it's one of the benefits a curated set of links provides.

The affirmations

Here are the things I've seen and learned. Each of these will be expanded in its own section.

• You have things to write about.
• Your perspective matters.
• You are good enough.
• Posts don't have to be novel.
• People will read it.
• Mistakes are okay!
• It's okay to ask for things.
• You can get started quickly.
• You can write on a schedule.

Despite the odds, cassette tapes are making a comeback. And one family-owned company in Springfield, Missouri is a leader in the revival.

Hopefully this also means that players are also making a comeback because you need something to play them on. Having visited a museum recently full of old radio recordings on cassette tapes, I was sad to find out I couldn't listen to them because the tape player was broken. It's hard enough finding a decent MP3 player, I'm sure it must be just as hard if not harder finding a tape player.

In the second part of the ownership series in Mojo, we built on the mental model developed in the first part and provided practical examples to illustrate how ownership works in Mojo. We covered the different kinds of values (BValue, LValue, and RValue) and how they propagate through expressions. We also explained the function argument conventions (borrowed, inout, owned) and demonstrated how these conventions help manage memory safely and efficiently. We concluded with three fundamental rules:

• Rule 1: Owned arguments take RValue on the caller side but are LValue on the callee side.
• Rule 2: Owned arguments own the type if the transfer operator ^ is used; otherwise, they copy the type if it is Copyable.
• Rule 3: Copy operations are optimized to move operations if the type is Copyable and Movable and isn’t used anymore, reducing unnecessary overhead.
  
  Lastly, we emphasized that the main goals of ownership in Mojo are:
  
  
• Memory Safety: Enforcing exclusive ownership and proper lifetimes to prevent memory errors such as use-after-free and double-free.
• Performance Optimization: Converting unnecessary copy operations into move operations to reduce overhead and enhance performance.
• Ease of Use: Automating memory management through ownership rules and the transfer operator, simplifying development.
• Compile-Time Guarantees: Providing strong compile-time guarantees through type-checking and dataflow lifetime analysis, catching errors early in the development process.

I’m proud to share the first post in a series from our friends, The Encryptids—the rarely-seen enigmas who inspire campfire lore. But this time, they’re spilling secrets about how they survive this ever-digital world. We begin by checking in with the legendary Bigfoot de la Sasquatch...

People say I'm the most famous of The Encryptids, but sometimes I don't want the spotlight. They all want a piece of me: exes, ad trackers, scammers, even the government. A picture may be worth a thousand words, but my digital profile is worth cash (to skeezy data brokers). I can’t hit a city block without being captured by doorbell cameras, CCTV, license plate readers, and a maze of street-level surveillance. It can make you want to give up on privacy altogether. Honey, no. Why should you have to hole up in some dank, busted forest for freedom and respect? You don’t.

Privacy isn't about hiding. It's about revealing what you want to who you want on your terms. It's your basic right to dignity.

Today we report a significant advance in understanding the inner workings of AI models. We have identified how millions of concepts are represented inside Claude Sonnet, one of our deployed large language models. This is the first ever detailed look inside a modern, production-grade large language model. This interpretability discovery could, in future, help us make AI models safer.

Previously, we made some progress matching patterns of neuron activations, called features, to human-interpretable concepts. We used a technique called "dictionary learning", borrowed from classical machine learning, which isolates patterns of neuron activations that recur across many different contexts. In turn, any internal state of the model can be represented in terms of a few active features instead of many active neurons. Just as every English word in a dictionary is made by combining letters, and every sentence is made by combining words, every feature in an AI model is made by combining neurons, and every internal state is made by combining features.

In October 2023, we reported success applying dictionary learning to a very small "toy" language model and found coherent features corresponding to concepts like uppercase text, DNA sequences, surnames in citations, nouns in mathematics, or function arguments in Python code.

We used the same scaling law philosophy that predicts the performance of larger models from smaller ones to tune our methods at an affordable scale before launching on Sonnet.

We successfully extracted millions of features from the middle layer of Claude 3.0 Sonnet, (a member of our current, state-of-the-art model family, currently available on claude.ai), providing a rough conceptual map of its internal states halfway through its computation. This is the first ever detailed look inside a modern, production-grade large language model.

Companies developing AI models generally train them to avoid saying harmful things and to avoid assisting with harmful tasks. The goal of this is to train models to behave in ways that are "harmless". But when we think of the character of those we find genuinely admirable, we don’t just think of harm avoidance. We think about those who are curious about the world, who strive to tell the truth without being unkind, and who are able to see many sides of an issue without becoming overconfident or overly cautious in their views. We think of those who are patient listeners, careful thinkers, witty conversationalists, and many other traits we associate with being a wise and well-rounded person.

AI models are not, of course, people. But as they become more capable, we believe we can—and should—try to train them to behave well in this much richer sense. Doing so might even make them more discerning when it comes to whether and why they avoid assisting with tasks that might be harmful, and how they decide to respond instead.

Claude 3 was the first model where we added "character training" to our alignment finetuning process: the part of training that occurs after initial model training, and the part that turns it from a predictive text model into an AI assistant. The goal of character training is to make Claude begin to have more nuanced, richer traits like curiosity, open-mindedness, and thoughtfulness.

Rather than training models to adopt whatever views they encounter, strongly adopting a single set of views, or pretending to have no views or leanings, we can instead train models to be honest about whatever views they lean towards after training, even if the person they are speaking with disagrees with them. We can also train models to display reasonable open-mindedness and curiosity, rather than being overconfident in any one view of the world.

In order to steer Claude’s character and personality, we made a list of many character traits we wanted to encourage the model to have...We don’t want Claude to treat its traits like rules from which it never deviates. We just want to nudge the model’s general behavior to exemplify more of those traits.

Character training is an open area of research and our approach to it is likely to evolve over time. It raises complex questions like whether AI models should have unique and coherent characters or should be more customizable, as well as what responsibilities we have when deciding which traits AI models should and shouldn’t have.

Ultravox is a new kind of multimodal LLM that can understand text as well as human speech, without the need for a separate Audio Speech Recognition (ASR) stage. Building on research like AudioLM, SeamlessM4T, Gazelle, SpeechGPT, and others, we've extended Meta's Llama 3 model with a multimodal projector that converts audio directly into the high-dimensional space used by Llama 3. This direct coupling allows Ultravox to respond much more quickly than systems that combine separate ASR and LLM components. In the future this will also allow Ultravox to natively understand the paralinguistic cues of timing and emotion that are omnipresent in human speech.

The current version of Ultravox (v0.1), when invoked with audio content, has a time-to-first-token (TTFT) of approximately 200ms, and a tokens-per-second rate of ~100, all using a Llama 3 8B backbone. While quite fast, we believe there is considerable room for improvement in these numbers. We look forward to working with LLM hosting providers to deliver state-of-the-art performance for Ultravox.

Ultravox currently takes in audio and emits streaming text. As we evolve the model, we'll train it to be able to emit a stream of speech tokens that can then be converted directly into raw audio by an appropriate unit vocoder. We're interested in working with interested parties to build this functionality!

We set out to build Private Cloud Compute with a set of core requirements:

1. Stateless computation on personal user data. ...we want a strong form of stateless data processing where personal data leaves no trace in the PCC system.
2. Enforceable guarantee. Security and privacy guarantees are strongest when they are entirely technically enforceable, which means it must be possible to constrain and analyze all the components that critically contribute to the guarantees of the overall Private Cloud Compute system.
3. No privileged runtime access. Private Cloud Compute must not contain privileged interfaces that would enable Apple’s site reliability staff to bypass PCC privacy guarantees, even when working to resolve an outage or other severe incident.
4. Non-targetability. An attacker should not be able to attempt to compromise personal data that belongs to specific, targeted Private Cloud Compute users without attempting a broad compromise of the entire PCC system.
5. Verifiable transparency. Security researchers need to be able to verify, with a high degree of confidence, that our privacy and security guarantees for Private Cloud Compute match our public promises.

Apple Intelligence is comprised of multiple highly-capable generative models that are specialized for our users’ everyday tasks, and can adapt on the fly for their current activity. The foundation models built into Apple Intelligence have been fine-tuned for user experiences such as writing and refining text, prioritizing and summarizing notifications, creating playful images for conversations with family and friends, and taking in-app actions to simplify interactions across apps.

In the following overview, we will detail how two of these models — a ~3 billion parameter on-device language model, and a larger server-based language model available with Private Cloud Compute and running on Apple silicon servers — have been built and adapted to perform specialized tasks efficiently, accurately, and responsibly. These two foundation models are part of a larger family of generative models created by Apple to support users and developers; this includes a coding model to build intelligence into Xcode, as well as a diffusion model to help users express themselves visually, for example, in the Messages app.

Pre-training

Our foundation models are trained on Apple's AXLearn framework, an open-source project we released in 2023. It builds on top of JAX and XLA, and allows us to train the models with high efficiency and scalability on various training hardware and cloud platforms, including TPUs and both cloud and on-premise GPUs. We used a combination of data parallelism, tensor parallelism, sequence parallelism, and Fully Sharded Data Parallel (FSDP) to scale training along multiple dimensions such as data, model, and sequence length.

Post-training

We find that data quality is essential to model success, so we utilize a hybrid data strategy in our training pipeline, incorporating both human-annotated and synthetic data, and conduct thorough data curation and filtering procedures. We have developed two novel algorithms in post-training: (1) a rejection sampling fine-tuning algorithm with teacher committee, and (2) a reinforcement learning from human feedback (RLHF) algorithm with mirror descent policy optimization and a leave-one-out advantage estimator. We find that these two algorithms lead to significant improvement in the model’s instruction-following quality.

Optimization

Both the on-device and server models use grouped-query-attention. We use shared input and output vocab embedding tables to reduce memory requirements and inference cost. These shared embedding tensors are mapped without duplications. The on-device model uses a vocab size of 49K, while the server model uses a vocab size of 100K, which includes additional language and technical tokens.

For on-device inference, we use low-bit palletization, a critical optimization technique that achieves the necessary memory, power, and performance requirements. To maintain model quality, we developed a new framework using LoRA adapters that incorporates a mixed 2-bit and 4-bit configuration strategy — averaging 3.5 bits-per-weight — to achieve the same accuracy as the uncompressed models.

Additionally, we use an interactive model latency and power analysis tool, Talaria, to better guide the bit rate selection for each operation. We also utilize activation quantization and embedding quantization, and have developed an approach to enable efficient Key-Value (KV) cache update on our neural engines.

Model Adaptation

Our foundation models are fine-tuned for users’ everyday activities, and can dynamically specialize themselves on-the-fly for the task at hand...

We represent the values of the adapter parameters using 16 bits, and for the ~3 billion parameter on-device model, the parameters for a rank 16 adapter typically require 10s of megabytes. The adapter models can be dynamically loaded, temporarily cached in memory, and swapped — giving our foundation model the ability to specialize itself on the fly for the task at hand while efficiently managing memory and guaranteeing the operating system's responsiveness.

Performance and Evaluation

We compare our models with both open-source models (Phi-3, Gemma, Mistral, DBRX) and commercial models of comparable size (GPT-3.5-Turbo, GPT-4-Turbo)1. We find that our models are preferred by human graders over most comparable competitor models. On this benchmark, our on-device model, with ~3B parameters, outperforms larger models including Phi-3-mini, Mistral-7B, and Gemma-7B. Our server model compares favorably to DBRX-Instruct, Mixtral-8x22B, and GPT-3.5-Turbo while being highly efficient.

To further evaluate our models, we use the Instruction-Following Eval (IFEval) benchmark to compare their instruction-following capabilities with models of comparable size. The results suggest that both our on-device and server model follow detailed instructions better than the open-source and commercial models of comparable size.

Apple today introduced Apple Intelligence, the personal intelligence system for iPhone, iPad, and Mac that combines the power of generative models with personal context to deliver intelligence that’s incredibly useful and relevant.

Apple Intelligence unlocks new ways for users to enhance their writing and communicate more effectively. With brand-new systemwide Writing Tools built into iOS 18, iPadOS 18, and macOS Sequoia, users can rewrite, proofread, and summarize text nearly everywhere they write...

Apple Intelligence powers exciting image creation capabilities to help users communicate and express themselves in new ways. With Image Playground, users can create fun images in seconds, choosing from three styles: Animation, Illustration, or Sketch...All images are created on device, giving users the freedom to experiment with as many images as they want.

Powered by Apple Intelligence, Siri becomes more deeply integrated into the system experience. With richer language-understanding capabilities, Siri is more natural, more contextually relevant, and more personal, with the ability to simplify and accelerate everyday tasks.

A cornerstone of Apple Intelligence is on-device processing, and many of the models that power it run entirely on device. To run more complex requests that require more processing power, Private Cloud Compute extends the privacy and security of Apple devices into the cloud to unlock even more intelligence.

Apple is integrating ChatGPT access into experiences within iOS 18, iPadOS 18, and macOS Sequoia, allowing users to access its expertise — as well as its image- and document-understanding capabilities — without needing to jump between tools.

I always look forward to watching these recap videos from the Verge.

The ability of LLMs to execute commands through plain language (e.g. English) has enabled agentic systems that can complete a user query by orchestrating the right set of tools...recent multi-modal efforts such as the GPT-4o or Gemini-1.5 model, has expanded the realm of possibilities with AI agents. While this is quite exciting, the large model size and computational requirements of these models often requires their inference to be performed on the cloud. This can create several challenges for their widespread adoption. First and foremost, uploading data such as video, audio, or text documents to a third party vendor on the cloud, can result in privacy issues. Second, this requires cloud/Wi-Fi connectivity which is not always possible...latency could also be an issue as uploading large amounts of data to the cloud and waiting for the response could slow down response time, resulting in unacceptable time-to-solution. These challenges could be solved if we deploy the LLM models locally at the edge.

...current LLMs like GPT-4o or Gemini-1.5 are too large for local deployment. One contributing factor is that a lot of the model size ends up memorizing general information about the world into its parametric memory which may not be necessary for a specialized downstream application.

...this leads to an intriguing research question:

Can a smaller language model with significantly less parametric memory emulate such emergent ability of these larger language models?

Achieving this would significantly reduce the computational footprint of agentic systems and thus enable efficient and privacy-preserving edge deployment. Our study demonstrates that this is feasible for small language models through training with specialized, high-quality data that does not require recalling generic world knowledge.

Such a system could particularly be useful for semantic systems where the AI agent’s role is to understand the user query in natural language and, instead of responding with a ChatGPT-type question answer response, orchestrate the right set of tools and APIs to accomplish the user’s command. For example, in a Siri-like application, a user may ask a language model to create a calendar invite with particular attendees. If a predefined script for creating calendar items already exists, the LLM simply needs to learn how to invoke this script with the correct input arguments (such as attendees’ email addresses, event title, and time). This process does not require recalling/memorization of world knowledge from sources like Wikipedia, but rather requires reasoning and learning to call the right functions and to correctly orchestrate them.

Our goal is to develop Small Language Models (SLM) that are capable of complex reasoning that could be deployed securely and privately at the edge. Here we will discuss the research directions that we are pursuing to that end. First, we discuss how we can enable small open-source models to perform accurate function calling, which is a key component of agentic systems. It turns out that off-the-shelf small models have very low function calling capabilities. We discuss how we address this by systematically curating high-quality data for function calling, using a specialized Mac assistant agent as our driving application. We then show that fine-tuning the model on this high quality curated dataset, can enable SLMs to even exceed GPT-4-Turbo’s function calling performance. We then show that this could be further improved and made efficient through a new Tool RAG method. Finally, we show how the final models could be deployed efficiently at the edge with real time responses.

Great performance.

This document aims to serve as a handbook for learning the key concepts underlying modern artificial intelligence systems. Given the speed of recent development in AI, there really isn’t a good textbook-style source for getting up-to-speed on the latest-and-greatest innovations in LLMs or other generative models, yet there is an abundance of great explainer resources (blog posts, videos, etc.) for these topics scattered across the internet. My goal is to organize the “best” of these resources into a textbook-style presentation, which can serve as a roadmap for filling in the prerequisites towards individual AI-related learning goals. My hope is that this will be a “living document”, to be updated as new innovations and paradigms inevitably emerge, and ideally also a document that can benefit from community input and contribution. This guide is aimed at those with a technical background of some kind, who are interested in diving into AI either out of curiosity or for a potential career. I’ll assume that you have some experience with coding and high-school level math, but otherwise will provide pointers for filling in any other prerequisites.

Cool project.

Turn a fresh Ubuntu installation into a fully-configured, beautiful, and modern web development system by running a single command.

Omakub is an opinionated take on what Linux can be at its best.

Omakub includes a curated set of applications and tools that one might discover through hours of watching YouTube, reading blogs, or just stumbling around Linux internet. All so someone coming straight from a platform like Windows or the Mac can immediately start enjoying a ready-made system, without having to do any configuration and curation legwork at all.

I'm guilty of owning way too many domains but this short-hand (lqdev.me) and other permadomain (luisquintanilla.me) are the ones I use mostly.

A potential happy medium I've seen is using subdomains (i.e. project.lqdev.me).

Jim Nielsen has great examples of how he's doing that today. Though that's an older post, he also addressed it most recently in the post Domain Sins of My Youth.

That somehow feels better than the URL you get when you use free hosting like GitHub Pages. As a bonus, it's already tied to your identity (if using your personal domain). Also, you save some money.

There's a few ways I practice this today. In addition to the previously mentioned domains, I also own lqdev.tech. Services and projects I host live there.

For example:

• Mastodon Instance (toot.lqdev.tech)
• Matrix server (matrix.lqdev.tech)
• Webmentions service (webmentions.lqdev.tech)

So far it's been working well.

This is cool! The Micro.blog premium plan has tons of great features as well.

Six years ago, we launched our $10/month plan with podcast hosting. Since then we’ve added several big features to the plan...

Today, I want to bring the podcast feature to more people, so we’re moving it down to the standard $5/month plan.

...the TLDR is that we're training a 12-layer GPT-2 (124M), from scratch, on 10B tokens of FineWeb, with max sequence length of 1024 tokens.

The 124M model is the smallest model in the GPT-2 series released by OpenAI in 2019, and is actually quite accessible today, even for the GPU poor. With llm.c, which is quite efficient at up to ~60% model flops utilization, reproducing this model on one 8X A100 80GB SXM node takes ~90 minutes. For example, on Lambda this node goes for ~$14/hr, so the total cost of reproducing this model today is about $20. You can train the model with a single GPU too, it would just take proportionally longer (e.g. ~4-24 hours depending on the GPU).

21 years since Mike and I did the first release of WordPress, forking Michel’s work on b2/cafélog.

I’ve been thinking a lot about elements that made WordPress successful in its early years that we should keep in mind as we build this year and beyond. Here’s 11 opinions:

1. Simple things should be easy and intuitive, and complex things possible.
   
   ...
2. Wikis are amazing, and our documentation should be wiki-easy to edit.
   
   ...
3. It’s important that we all do support, go to meetups and events, anything we can to stay close to regular end-users of what we make.

Congrats to WordPress and the team on 21 years. I think the following are some good elements to keep building on.

Like many in the comments, I like the idea of this device and what it's aiming to do.

The "LivePaper" display technology seems interesting.

At $729 though, it seems a little steep. I paid a fraction of that for my Onyx Boox Nova Air 2 and I have most of the functionality of this device. As an e-reader, note-taking device, and tablet that can install any Android app, it works perfectly fine for my use cases and I'm very happy with it.

Still, I'm interested in seeing whether this takes off or whether it ends up being another Rabbit R1 or Humane Ai Pin.

🙋

Great post.

One of the challenges I've found, even when using appliances like those from hosting providers like Linode or tools like YunoHost is connecting them to your domain.

Usually, that's specific to your domain name provider and usually a manual process.

In cases where that's easy, you're often overcharged for the convenience. Compared to "free" social media and publishing websites, it makes using your own domain a less desirable option.

Maybe advocating for a multi-staged approach like IndieWebify.Me could make the journey more approachable. More importantly, making it a journey, rather than a destination could help with meeting folks where they are and guide them closer towards their goals.

In any case, the proposed list of goals seem like a great start towards helping people create their own place on the web.

The face of one of the defining memes of the 2010s, the doge meme, died on Friday. Kabosu, the shiba inu with the knowing face that launched a million internet jokes, was 18 when she died

I never knew I needed this in my life.

Between this article and The Dumbphone Boom is Real, I'm seeing more publications on the subject. Far from a comeback, but still cool to see.

2024 just keeps getting better in terms of new music releases.

Kamasi Washington has a new album coming out this week called Fearless Movement.

The most recent single, Dream State, with Andre 3000 is great.

Will Stults spent too much time on his iPhone, doom-scrolling the site formerly known as Twitter and tweeting angrily at Elon Musk as if the billionaire would actually notice. Stults’s partner, Daisy Krigbaum, was addicted to Pinterest and YouTube, bingeing videos on her iPhone before going to sleep. Two years ago, they both tried Apple’s Screen Time restriction tool and found it too easy to disable, so the pair decided to trade out their iPhones for more low-tech devices. They’d heard about so-called dumbphones, which lacked the kinds of bells and whistles—a high-resolution screen, an app store, a video camera—that made smartphones so addictive. But they found the process of acquiring one hard to navigate. “The information on it was kind of disparate and hard to get to. A lot of people who know the most about dumbphones spend the least time online,” Krigbaum said. A certain irony presented itself: figuring out a way to be less online required aggressive online digging.

The growing dumbphone fervor may be motivated, in part, by the discourse around child safety online. Parents are increasingly confronted with evidence that sites like Instagram and TikTok intentionally try to hook their children. Using those sites can increase teens’ anxiety and lower their self-esteem, according to some studies, and smartphones make it so that kids are logged on constantly. Why should this situation be any healthier for adults? After almost two decades with iPhones, the public seems to be experiencing a collective ennui with digital life. So many hours of each day are lived through our portable, glowing screens, but the Internet isn’t even fun anymore. We lack the self-control to wean ourselves off, so we crave devices that actively prevent us from getting sucked into them. That means opting out of the prevailing technology and into what Cal Newport, a contributing writer for The New Yorker, has called a more considered “digital minimalism.”

While dumbphones aren't a cure-all for unhealthy technology habits, as a dumbphone user, I can relate to the frustrations that come from the lack of device availability and support. Even when new devices hit the market, they tend to be targeted towards non-US markets.

Today, the Snowflake AI Research Team is thrilled to introduce Snowflake Arctic, a top-tier enterprise-focused LLM that pushes the frontiers of cost-effective training and openness. Arctic is efficiently intelligent and truly open.

• Efficiently Intelligent: Arctic excels at enterprise tasks such as SQL generation, coding and instruction following benchmarks even when compared to open source models trained with significantly higher compute budgets. In fact, it sets a new baseline for cost-effective training to enable Snowflake customers to create high-quality custom models for their enterprise needs at a low cost.
  
  
• Truly Open: Apache 2.0 license provides ungated access to weights and code. In addition, we are also open sourcing all of our data recipes and research insights.
  
  Snowflake Arctic is available from Hugging Face, NVIDIA API catalog and Replicate today or via your model garden or catalog of choice, including Snowflake Cortex, Amazon Web Services (AWS), Microsoft Azure, Lamini, Perplexity and Together over the coming days.

PBS is making child edutainment classics like Zoboomafoo, Mister Rogers’ Neighborhood, and Reading Rainbow available for free on a new ‘PBS Retro’ channel on Roku.

This is cool! Although I'm kind of bummed the stuff I grew up watching is now considered "retro".

The internet has become an extractive and fragile monoculture. But we can revitalize it using lessons learned by ecologists.

Our online spaces are not ecosystems, though tech firms love that word. They’re plantations; highly concentrated and controlled environments...

We all know this. We see it each time we reach for our phones. But what most people have missed is how this concentration reaches deep into the internet’s infrastructure — the pipes and protocols, cables and networks, search engines and browsers. These structures determine how we build and use the internet, now and in the future.

They’ve concentrated into a series of near-planetary duopolies.

Two kinds of everything may be enough to fill a fictional ark and repopulate a ruined world, but can’t run an open, global “network of networks” where everyone has the same chance to innovate and compete.

The internet made the tech giants possible. Their services have scaled globally, via its open, interoperable core. But for the past decade, they’ve also worked to enclose the varied, competing and often open-source or collectively provided services the internet is built on into their proprietary domains. Although this improves their operational efficiency, it also ensures that the flourishing conditions of their own emergence aren’t repeated by potential competitors. For tech giants, the long period of open internet evolution is over. Their internet is not an ecosystem. It’s a zoo.

Up close, internet concentration seems too intricate to untangle; from far away, it seems too difficult to deal with. But what if we thought of the internet not as a doomsday “hyperobject,” but as a damaged and struggling ecosystem facing destruction? What if we looked at it not with helpless horror at the eldritch encroachment of its current controllers, but with compassion, constructiveness and hope?

Rewilding “aims to restore healthy ecosystems by creating wild, biodiverse spaces,” according to the International Union for Conservation of Nature. More ambitious and risk-tolerant than traditional conservation, it targets entire ecosystems to make space for complex food webs and the emergence of unexpected interspecies relations. It’s less interested in saving specific endangered species. Individual species are just ecosystem components, and focusing on components loses sight of the whole. Ecosystems flourish through multiple points of contact between their many elements, just like computer networks. And like in computer networks, ecosystem interactions are multifaceted and generative.

Whatever we do, the internet isn’t returning to old-school then-common interfaces like FTP and Gopher, or organizations operating their own mail servers again instead of off-the-shelf solutions like G-Suite. But some of what we need is already here, especially on the web. Look at the resurgence of RSS feeds, email newsletters and blogs, as we discover (yet again) that relying on one app to host global conversations creates a single point of failure and control. New systems are growing, like the Fediverse with its federated islands, or Bluesky with algorithmic choice and composable moderation.

We don’t know what the future holds. Our job is to keep open as much opportunity as we can, trusting that those who come later will use it. Instead of setting purity tests for which kind of internet is most like the original, we can test changes against the values of the original design. Do new standards protect the network’s “generality,” i.e. its ability to support multiple uses, or is functionality limited to optimize efficiency for the biggest tech firms?

...our internet took off because it was designed as a general-purpose network, built to connect anyone.

Our internet was built to be complex and unbiddable, to do things we cannot yet imagine.

Internet infrastructure is a degraded ecosystem, but it’s also a built environment, like a city. Its unpredictability makes it generative, worthwhile and deeply human.

We need to stop thinking of internet infrastructure as too hard to fix. It’s the underlying system we use for nearly everything we do.

Rewilding the internet connects and grows what people are doing across regulation, standards-setting and new ways of organizing and building infrastructure, to tell a shared story of where we want to go. It’s a shared vision with many strategies. The instruments we need to shift away from extractive technological monocultures are at hand or ready to be built.

Calculus Made Easy is a book on calculus originally published in 1910 by Silvanus P. Thompson, considered a classic and elegant introduction to the subject.

• Project Gutenberg PDF

I’ve been meaning to write some kind of Important Thinkpiece™ on the glory days of the early internet, but every time I sit down to do it, I find another, better piece that someone else has already written. So for now, here’s a collection of articles that to some degree answer the question “Why have a personal website?” with “Because it’s fun, and the internet used to be fun.”

This is a great catalog of posts about the personal web. Courtesy of Rachel Kwon

This is a site encouraging non-technical people and organisations to create their own online services such as websites, social networks, personal clouds, instant messaging etc.

Large language models (LLMs) have revolutionized a wide range of tasks and applications that were previously reliant on manually crafted machine learning (ML) solutions, streamlining through automation. However, despite these advances, a notable challenge persists: the need for extensive prompt engineering to adapt these models to new tasks. New generations of language models like GPT-4 and Mixtral 8x7B advance the capability to process long input texts. This progress enables the use of longer inputs, providing richer context and detailed instructions to language models. A common technique that uses this enhanced capacity is the Retrieval Augmented Generation (RAG) approach. RAG dynamically incorporates information into the prompt based on the specific input example.

To address these challenges, we developed the Structure-Aware Multi-objective Metaprompt Optimization (SAMMO) framework. SAMMO is a new open-source tool that streamlines the optimization of prompts, particularly those that combine different types of structural information like in the RAG example above. It can make structural changes, such as removing entire components or replacing them with different ones. These features enable AI practitioners and researchers to efficiently refine their prompts with little manual effort.

Central to SAMMO’s innovation is its approach to treating prompts not just as static text inputs but as dynamic, programmable entities—metaprompts. SAMMO represents these metaprompts as function graphs, where individual components and substructures can be modified to optimize performance, similar to the optimization process that occurs during traditional program compilation.

The following key features contribute to SAMMO’s effectiveness:

• Structured optimization: Unlike current methods that focus on text-level changes, SAMMO focuses on optimizing the structure of metaprompts. This granular approach facilitates precise modifications and enables the straightforward integration of domain knowledge, for instance, through rewrite operations targeting specific stylistic objectives.
  
  
• Multi-objective search: SAMMO’s flexibility enables it to simultaneously address multiple objectives, such as improving accuracy and computational efficiency. Our paper illustrates how SAMMO can be used to compress prompts without compromising their accuracy.
  
  
• General purpose application: SAMMO has proven to deliver significant performance improvements across a variety of tasks, including instruction tuning, RAG, and prompt compression.

Reading this post and attached image brings back so many memories.

I remember in the last few years of Blockbuster and early days of Netflix, for about $20/month you could rent unlimited movies (I think it was up to three at a time).

If you binge watched them or weren't happy with the choices you made, you could just drive down to your local store, return them, and grab a new set of movies.

Initially you had a return period, but towards the end, there were none so you basically got to keep some movies as long as you liked.

Good times.

In 2024, Ghost is adopting ActivityPub and connecting with other federated platforms across the web.

This means that, soon, Ghost publishers will be able to follow, like and interact with one another in the same way that you would normally do on a social network — but on your own website.

The difference, of course, is that you’ll also be able to follow, like, and interact with users on Mastodon, Threads, Flipboard, Buttondown, WriteFreely, Tumblr, WordPress, PeerTube, Pixelfed... or any other platform that has adopted ActivityPub, too. You don’t need to limit yourself to following people who happen to use the same platform as you.

For the past few years the choice has been difficult. Either participate in closed networks at the mercy of algorithms, or set up an independent website at the expense of your growth.

Email gave us private messaging technology that isn’t owned by a single company.

ActivityPub is doing the same for social technology.

The open web is coming back, and with it returns diversity. You can both publish independently and grow faster than ever before with followers from all over the world & the web.

I can't express how much I love this. Personally I don't use Ghost, but given platforms like WordPress and now Ghost are adding support for ActivityPub, it empowers people to build their own platforms.

That said, this still doesn't address the challenges of building your own website, which as mentioned in the post are one of the appealing aspects of current closed networks.

Still though, there is a vast number of creators, businesses, and company websites or blogs that can benefit from this today. When paired with RSS, it gives people choice and autonomy in how they create and consume content as I mentioned in a previous post, Rediscovering the RSS protocol

A JAX research toolkit for building, editing, and visualizing neural networks.

Penzai is a JAX library for writing models as legible, functional pytree data structures, along with tools for visualizing, modifying, and analyzing them. Penzai focuses on making it easy to do stuff with models after they have been trained, making it a great choice for research involving reverse-engineering or ablating model components, inspecting and probing internal activations, performing model surgery, debugging architectures, and more. (But if you just want to build and train a model, you can do that too!)

Starting today, Phi-3-mini, a 3.8B language model is available on Microsoft Azure AI Studio, Hugging Face, and Ollama.

• Phi-3-mini is available in two context-length variants—4K and 128K tokens. It is the first model in its class to support a context window of up to 128K tokens, with little impact on quality.
• It is instruction-tuned, meaning that it’s trained to follow different types of instructions reflecting how people normally communicate. This ensures the model is ready to use out-of-the-box.
• It is available on Azure AI to take advantage of the deploy-eval-finetune toolchain, and is available on Ollama for developers to run locally on their laptops.
• It has been optimized for ONNX Runtime with support for Windows DirectML along with cross-platform support across graphics processing unit (GPU), CPU, and even mobile hardware.
• It is also available as an NVIDIA NIM microservice with a standard API interface that can be deployed anywhere. And has been optimized for NVIDIA GPUs.
  
  In the coming weeks, additional models will be added to Phi-3 family to offer customers even more flexibility across the quality-cost curve. Phi-3-small (7B) and Phi-3-medium (14B) will be available in the Azure AI model catalog and other model gardens shortly.

Build the future of AI with Meta Llama 3

Now available with both 8B and 70B pretrained and instruction-tuned versions to support a wide range of applications

Llama 3 models take data and scale to new heights. It’s been trained on our two recently announced custom-built 24K GPU clusters on over 15T token of data – a training dataset 7x larger than that used for Llama 2, including 4x more code. This results in the most capable Llama model yet, which supports a 8K context length that doubles the capacity of Llama 2.

While listening to KHOL earlier today, they were rebroadcasting a recording of A Todo Pulmon, a radio show from Radio Bilingue in Fresno, CA. Good stuff.

...today, we’re happy to announce that Standard Notes will also join us to advance our shared mission.

Both Proton and Standard Notes share a strong commitment to our communities, so Standard Notes will remain open source, freely available, and fully supported. Prices are not changing, and if you have a current subscription to Standard Notes, it will continue to be honored. Proton aspires to do the right thing and be a responsible home for open-source projects, and just as we did with SimpleLogin, we are committed to preserving what makes Standard Notes special and much loved.

In the coming months, we hope to find ways to make Standard Notes more easily accessible to the Proton community. This way, in addition to protecting your email, calendar, files, passwords, and online activity, you can also protect your notes.

This is another exciting acquisition! I mainly use org-mode in Emacs for note taking. However, I love the ecosystem Proton is building with their security and privacy focused set of collaborative software offerings.

Recurrent neural networks (RNNs) have fast inference and scale efficiently on long sequences, but they are difficult to train and hard to scale. We propose Hawk, an RNN with gated linear recurrences, and Griffin, a hybrid model that mixes gated linear recurrences with local attention. Hawk exceeds the reported performance of Mamba on downstream tasks, while Griffin matches the performance of Llama-2 despite being trained on over 6 times fewer tokens. We also show that Griffin can extrapolate on sequences significantly longer than those seen during training. Our models match the hardware efficiency of Transformers during training, and during inference they have lower latency and significantly higher throughput. We scale Griffin up to 14B parameters, and explain how to shard our models for efficient distributed training.

How can we detect if copyrighted content was used in the training process of a language model, considering that the training data is typically undisclosed? We are motivated by the premise that a language model is likely to identify verbatim excerpts from its training text. We propose DE-COP, a method to determine whether a piece of copyrighted content was included in training. DE-COP's core approach is to probe an LLM with multiple-choice questions, whose options include both verbatim text and their paraphrases. We construct BookTection, a benchmark with excerpts from 165 books published prior and subsequent to a model's training cutoff, along with their paraphrases. Our experiments show that DE-COP surpasses the prior best method by 9.6% in detection performance (AUC) on models with logits available. Moreover, DE-COP also achieves an average accuracy of 72% for detecting suspect books on fully black-box models where prior methods give ≈ 4% accuracy. Our code and datasets are available at this https URL

Repo

Text-to-SQL models can generate a list of candidate SQL queries, and the best query is often in the candidate list, but not at the top of the list. An effective re-rank method can select the right SQL query from the candidate list and improve the model's performance. Previous studies on code generation automatically generate test cases and use them to re-rank candidate codes. However, automatic test case generation for text-to-SQL is an understudied field. We propose an automatic test case generation method that first generates a database and then uses LLMs to predict the ground truth, which is the expected execution results of the ground truth SQL query on this database. To reduce the difficulty for LLMs to predict, we conduct experiments to search for ways to generate easy databases for LLMs and design easy-to-understand prompts. Based on our test case generation method, we propose a re-rank method to select the right SQL query from the candidate list. Given a candidate list, our method can generate test cases and re-rank the candidate list according to their pass numbers on these test cases and their generation probabilities. The experiment results on the validation dataset of Spider show that the performance of some state-of-the-art models can get a 3.6% improvement after applying our re-rank method.

RecurrentGemma is a family of open-weights Language Models by Google DeepMind, based on the novel Griffin architecture. This architecture achieves fast inference when generating long sequences by replacing global attention with a mixture of local attention and linear recurrences.

This repository contains the model implementation and examples for sampling and fine-tuning. We recommend most users adopt the Flax implementation, which is highly optimized. We also provide an un-optimized PyTorch implementation for reference.

Today, The Allen Institute for AI (AI2) has released OLMo 7B, a truly open, state-of-the-art large language model released alongside the pre-training data and training code. This empowers researchers and developers to use the best and open models to advance the science of language models collectively.

OLMo-7B on HuggingFace

LLM training in simple, pure C/CUDA. There is no need for 245MB of PyTorch or 107MB of cPython. For example, training GPT-2 (CPU, fp32) is ~1,000 lines of clean code in a single file. It compiles and runs instantly, and exactly matches the PyTorch reference implementation. I chose GPT-2 as the first working example because it is the grand-daddy of LLMs, the first time the modern stack was put together.

Beeper, the upstart messaging app that attempts to corral all your messaging services into one inbox, is being acquired by Automattic, the giant that runs Wordpress.com, Tumblr, and a number of other hugely popular web properties

This is exciting especially given some of the recent developments in the EU. What's most interesting to me is how Beeper leverages open protocols like Matrix and for bridging capabilities where possible to provide secure messaging.

With more people moving to smaller spaces to communicate with their communities, being able to do so in a single place without everyone being on the same platform like in the early days of the internet is a welcome development.

Additional coverage from the Beeper blog.

What we’re announcing today…

• No more waitlist – Beeper is now available to everyone!
• Beeper has been acquired by Automattic
• Our new Android app is out of beta
• We’re renaming Beeper Cloud → Beeper (sorry for the confusion)

and Matt Mullenweg's blog

Today the announcement went out that we’re combining the best technology from Beeper and Texts to create a great private, secure, and open source messaging client for people to have control of their communications. We’re going to use the Beeper brand, because it’s fun. This is not unlike how browsers have evolved, where solid tech and encryption on top of an open ecosystem has created untold value for humanity.

A lot of people are asking about iMessage on Android… I have zero interest in fighting with Apple, I think instead it’s best to focus on messaging networks that want more engagement from power-user clients. This is an area I’m excited to work on when I return from my sabbatical next month.

Retrieval-Augmented Generation (RAG) is essential for integrating external knowledge into Large Language Model (LLM) outputs. While the literature on RAG is growing, it primarily focuses on systematic reviews and comparisons of new state-of-the-art (SoTA) techniques against their predecessors, with a gap in extensive experimental comparisons. This study begins to address this gap by assessing various RAG methods' impacts on retrieval precision and answer similarity. We found that Hypothetical Document Embedding (HyDE) and LLM reranking significantly enhance retrieval precision. However, Maximal Marginal Relevance (MMR) and Cohere rerank did not exhibit notable advantages over a baseline Naive RAG system, and Multi-query approaches underperformed. Sentence Window Retrieval emerged as the most effective for retrieval precision, despite its variable performance on answer similarity. The study confirms the potential of the Document Summary Index as a competent retrieval approach. All resources related to this research are publicly accessible for further investigation through our GitHub repository ARAGOG (this https URL). We welcome the community to further this exploratory study in RAG systems.

GitHub repo

Outdoor nagivation app for long-distance trails

Bookmarking for reference.

I'm already subscribed to many of these websites and publications. However, there's several new ones I found that I think will eventually make their rotation into my blogroll.

New fine-tuning API features

Today, we’re introducing new features to give developers even more control over their fine-tuning jobs, including:

• Epoch-based Checkpoint Creation: Automatically produce one full fine-tuned model checkpoint during each training epoch, which reduces the need for subsequent retraining, especially in the cases of overfitting
• Comparative Playground: A new side-by-side Playground UI for comparing model quality and performance, allowing human evaluation of the outputs of multiple models or fine-tune snapshots against a single prompt
• Third-party Integration: Support for integrations with third-party platforms (starting with Weights and Biases this week) to let developers share detailed fine-tuning data to the rest of their stack
• Comprehensive Validation Metrics: The ability to compute metrics like loss and accuracy over the entire validation dataset instead of a sampled batch, providing better insight on model quality
• Hyperparameter Configuration: The ability to configure available hyperparameters from the Dashboard (rather than only through the API or SDK)
• Fine-Tuning Dashboard Improvements: Including the ability to configure hyperparameters, view more detailed training metrics, and rerun jobs from previous configurations

Expanding our Custom Models Program

• Assisted Fine-Tuning

Today, we are formally announcing our assisted fine-tuning offering as part of the Custom Model program. Assisted fine-tuning is a collaborative effort with our technical teams to leverage techniques beyond the fine-tuning API, such as additional hyperparameters and various parameter efficient fine-tuning (PEFT) methods at a larger scale. It’s particularly helpful for organizations that need support setting up efficient training data pipelines, evaluation systems, and bespoke parameters and methods to maximize model performance for their use case or task.

• Custom-Trained Model

In some cases, organizations need to train a purpose-built model from scratch that understands their business, industry, or domain. Fully custom-trained models imbue new knowledge from a specific domain by modifying key steps of the model training process using novel mid-training and post-training techniques. Organizations that see success with a fully custom-trained model often have large quantities of proprietary data—millions of examples or billions of tokens—that they want to use to teach the model new knowledge or complex, unique behaviors for highly specific use cases.

Command R+ is a state-of-the-art RAG-optimized model designed to tackle enterprise-grade workloads, and is available first on Microsoft Azure

Command R+, like our recently launched Command R model, features a 128k-token context window and is designed to offer best-in-class:

• Advanced Retrieval Augmented Generation (RAG) with citation to reduce hallucinations
• Multilingual coverage in 10 key languages to support global business operations
• Tool Use to automate sophisticated business processes

By encoding time series as a string of numerical digits, we can frame time series forecasting as next-token prediction in text. Developing this approach, we find that large language models (LLMs) such as GPT-3 and LLaMA-2 can surprisingly zero-shot extrapolate time series at a level comparable to or exceeding the performance of purpose-built time series models trained on the downstream tasks. To facilitate this performance, we propose procedures for effectively tokenizing time series data and converting discrete distributions over tokens into highly flexible densities over continuous values. We argue the success of LLMs for time series stems from their ability to naturally represent multimodal distributions, in conjunction with biases for simplicity, and repetition, which align with the salient features in many time series, such as repeated seasonal trends. We also show how LLMs can naturally handle missing data without imputation through non-numerical text, accommodate textual side information, and answer questions to help explain predictions. While we find that increasing model size generally improves performance on time series, we show GPT-4 can perform worse than GPT-3 because of how it tokenizes numbers, and poor uncertainty calibration, which is likely the result of alignment interventions such as RLHF.

Github Repo

IPEX-LLM is a PyTorch library for running LLM on Intel CPU and GPU (e.g., local PC with iGPU, discrete GPU such as Arc, Flex and Max) with very low latency

Reference resolution is an important problem, one that is essential to understand and successfully handle context of different kinds. This context includes both previous turns and context that pertains to non-conversational entities, such as entities on the user's screen or those running in the background. While LLMs have been shown to be extremely powerful for a variety of tasks, their use in reference resolution, particularly for non-conversational entities, remains underutilized. This paper demonstrates how LLMs can be used to create an extremely effective system to resolve references of various types, by showing how reference resolution can be converted into a language modeling problem, despite involving forms of entities like those on screen that are not traditionally conducive to being reduced to a text-only modality. We demonstrate large improvements over an existing system with similar functionality across different types of references, with our smallest model obtaining absolute gains of over 5% for on-screen references. We also benchmark against GPT-3.5 and GPT-4, with our smallest model achieving performance comparable to that of GPT-4, and our larger models substantially outperforming it.

Stable Audio 2.0 sets a new standard in AI-generated audio, producing high-quality, full tracks with coherent musical structure up to three minutes in length at 44.1kHz stereo.

The new model introduces audio-to-audio generation by allowing users to upload and transform samples using natural language prompts.

Stable Audio 2.0 was exclusively trained on a licensed dataset from the AudioSparx music library, honoring opt-out requests and ensuring fair compensation for creators.

Deadline reports that The Martian writer Drew Goddard has been tapped to pen and direct another Matrix movie executive produced by Lana Wachowski. Currently, the new film has no title or projected premiere date, and there’s been no announcement as to whether franchise stars like Keanu Reeves, Carrie-Anne Moss, Laurence Fishburne, Yahya Abdul-Mateen II, or Jessica Henwick will return.

Not sure how to feel about this, but I'll end up watching anyway.

Good article. I felt this way when Google Reader and a few other services were shut down.

That being said, this is kind of a good thing.

Luckily, there are plenty of good podcast apps out there, like Pocket Casts, Overcast, Antennapod, and even Apple Podcasts.

This line basically says it all. Podcasts, like blogging, continue to be an open ecosystem and where the saying,"wherever you get your podcasts", is still going strong.

We’re making it easier for people to experience the benefits of AI without needing to sign up.

We may use what you provide to ChatGPT to improve our models for everyone. If you’d like, you can turn this off through your Settings - whether you create an account or not.

We’ve also introduced additional content safeguards for this experience, such as blocking prompts and generations in a wider range of categories.

Bring back the Sidekick! Ayaneo Slide is probably the closest to this today. Would love to see a smaller version of it running on Windows on ARM-based Snapdragon processors.

Before the iPhone, before Android, before webOS, a revolutionary soap bar of a phone made it incredibly easy to get shit done. The Danger Hiptop, better known as the T-Mobile Sidekick, made the internet portable and affordable like no phone before.

Today, we are excited to advance our mission by open sourcing DBRX, a general purpose large language model (LLM) built by our Mosaic Research team that outperforms all established open source models on standard benchmarks. We believe that pushing the boundary of open source models enables generative AI for all enterprises that is customizable and transparent.

We are excited about DBRX for three distinct reasons. First, it handily beats open source models, such as, LLaMA2-70B, Mixtral, and Grok-1 on language understanding, programming, math, and logic...

Second, DBRX beats GPT-3.5 on most benchmarks...

Third, DBRX is a Mixture-of-Experts (MoE) model built on the MegaBlocks research and open source project, making the model extremely fast in terms of tokens/second.

Our one-step generator achieves comparable image quality with StableDiffusion v1.5 while being 30x faster.

Diffusion models are known to approximate the score function of the distribution they are trained on. In other words, an unrealistic synthetic image can be directed toward higher probability density region through the denoising process (see SDS). Our core idea is training two diffusion models to estimate not only the score function of the target real distribution, but also that of the fake distribution. We construct a gradient update to our generator as the difference between the two scores, essentially nudging the generated images toward higher realism as well as lower fakeness (see VSD). Our method is similar to GANs in that a critic is jointly trained with the generator to minimize a divergence between the real and fake distributions, but differs in that our training does not play an adversarial game that may cause training instability, and our critic can fully leverage the weights of a pretrained diffusion model. Combined with a simple regression loss to match the output of the multi-step diffusion model, our method outperforms all published few-step diffusion approaches, reaching 2.62 FID on ImageNet 64x64 and 11.49 FID on zero-shot COCO-30k, comparable to Stable Diffusion but orders of magnitude faster. Utilizing FP16 inference, our model generates images at 20 FPS on modern hardware.

Paper

We paid for the development of full text RSS feeds for Ghost-based publishers. Now we can offer them to our paid subscribers, and other Ghost sites can use the service too.

Our friends Anil Dash and Ernie Smith have recently written passionately and persuasively about the importance of RSS to the open web, and about how a technology that turns 25 years old this month remains both subversive and quite versatile. RSS-based distribution underpins a podcasting ecosystem that has allowed for shows to be distributed not just on Apple Podcasts but on Spotify, Google Podcasts, Pocket Casts, Overcast, and whatever other podcast player you might want to listen on. “Being able to say, ‘wherever you get your podcasts’ is a radical statement,” Dash wrote. “Because what it represents is the triumph of exactly the kind of technology that's supposed to be impossible: open, empowering tech that's not owned by any one company, that can't be controlled by any one company, and that allows people to have ownership over their work and their relationship with their audience.”

RSS has empowered podcasters, but that it needs a “creator economy rethink” for text.

Earlier today, Emad Mostaque resigned from his role as CEO of Stability AI and from his position on the Board of Directors of the company to pursue decentralized AI.

Foundation models, now powering most of the exciting applications in deep learning, are almost universally based on the Transformer architecture and its core attention module. Many subquadratic-time architectures such as linear attention, gated convolution and recurrent models, and structured state space models (SSMs) have been developed to address Transformers' computational inefficiency on long sequences, but they have not performed as well as attention on important modalities such as language. We identify that a key weakness of such models is their inability to perform content-based reasoning, and make several improvements. First, simply letting the SSM parameters be functions of the input addresses their weakness with discrete modalities, allowing the model to selectively propagate or forget information along the sequence length dimension depending on the current token. Second, even though this change prevents the use of efficient convolutions, we design a hardware-aware parallel algorithm in recurrent mode. We integrate these selective SSMs into a simplified end-to-end neural network architecture without attention or even MLP blocks (Mamba). Mamba enjoys fast inference (5× higher throughput than Transformers) and linear scaling in sequence length, and its performance improves on real data up to million-length sequences. As a general sequence model backbone, Mamba achieves state-of-the-art performance across several modalities such as language, audio, and genomics. On language modeling, our Mamba-3B model outperforms Transformers of the same size and matches Transformers twice its size, both in pretraining and downstream evaluation.

We announce today the release of Common Corpus on HuggingFace:

• Common Corpus is the largest public domain dataset released for training LLMs.
• Common Corpus includes 500 billion words from a wide diversity of cultural heritage initiatives.
• Common Corpus is multilingual and the largest corpus to date in English, French, Dutch, Spanish, German and Italian.
• Common Corpus shows it is possible to train fully open LLMs on sources without copyright concerns.

Welcome to the course that will teach you the most fascinating topic in game development: how to use powerful AI tools and models to create unique game experiences.

New AI models are revolutionizing the Game Industry in two impactful ways:

• On how we make games:
  • Generate textures using AI
  • Using AI voice actors for the voices.
    
    
• How we create gameplay:
  • Crafting smart Non-Playable Characters (NPCs) using large language models.
    
    This course will teach you:
    
    
• How to integrate AI models for innovative gameplay, featuring intelligent NPCs.
• How to use AI tools to help your game development pipeline.

In the same way that every little place in America used to have a printed newspaper, every little place in America could have an online local chronicle.

Broadly speaking, an online local chronicle is a collection of facts organized mostly in chronological order. The “pages” of the chronicle can be thought of as subsets of a community’s universal timeline of events. These online local chronicles could become the backbone of local news operations.

Nice project. Unfortunately, it's rare you get local news. I like publications / websites like Hoboken Girl and Block Club Chicago. I wish there were more of them in more cities and towns. I know they're there in some forms like Facebook Groups. Even better, it'd be great to have the websites for these publications be the main source of truth that then syndicated their content to the various platforms out there.

Quantization is a technique to reduce the computational and memory costs of evaluating Deep Learning Models by representing their weights and activations with low-precision data types like 8-bit integer (int8) instead of the usual 32-bit floating point (float32).

Today, we are excited to introduce quanto, a versatile pytorch quantization toolkit, that provides several unique features:

• available in eager mode (works with non-traceable models)
• quantized models can be placed on any device (including CUDA and MPS),
• automatically inserts quantization and dequantization stubs,
• automatically inserts quantized functional operations,
• automatically inserts quantized modules (see below the list of supported modules),
• provides a seamless workflow for a float model, going from a dynamic to a static quantized model,
• supports quantized model serialization as a state_dict,
• supports not only int8 weights, but also int2 and int4,
• supports not only int8 activations, but also float8.

Embeddings have become a pivotal means to represent complex, multi-faceted information about entities, concepts, and relationships in a condensed and useful format. Nevertheless, they often preclude direct interpretation. While downstream tasks make use of these compressed representations, meaningful interpretation usually requires visualization using dimensionality reduction or specialized machine learning interpretability methods. This paper addresses the challenge of making such embeddings more interpretable and broadly useful, by employing Large Language Models (LLMs) to directly interact with embeddings -- transforming abstract vectors into understandable narratives. By injecting embeddings into LLMs, we enable querying and exploration of complex embedding data. We demonstrate our approach on a variety of diverse tasks, including: enhancing concept activation vectors (CAVs), communicating novel embedded entities, and decoding user preferences in recommender systems. Our work couples the immense information potential of embeddings with the interpretative power of LLMs.

Experiment with different tokenizers (running locally in your browser). I really love doing this thing

Today we are releasing Stable Video 3D (SV3D), a generative model based on Stable Video Diffusion, advancing the field of 3D technology and delivering greatly improved quality and view-consistency.

This release features two variants: SV3D_u and SV3D_p. SV3D_u generates orbital videos based on single image inputs without camera conditioning. SV3D_p extends the capability by accommodating both single images and orbital views, allowing for the creation of 3D video along specified camera paths.

Stable Video 3D can be used now for commercial purposes with a Stability AI Membership. For non-commercial use, you can download the model weights on Hugging Face and view our research paper here.

Nvidia reveals Blackwell B200 GPU, the ‘world’s most powerful chip’ for AI

‘Built to democratize trillion-parameter AI.’

Nvidia says the new B200 GPU offers up to 20 petaflops of FP4 horsepower from its 208 billion transistors. Also, it says, a GB200 that combines two of those GPUs with a single Grace CPU can offer 30 times the performance for LLM inference workloads while also potentially being substantially more efficient. It “reduces cost and energy consumption by up to 25x” over an H100, says Nvidia.

In this work, we discuss building performant Multimodal Large Language Models (MLLMs). In particular, we study the importance of various architecture components and data choices. Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons. For example, we demonstrate that for large-scale multimodal pre-training using a careful mix of image-caption, interleaved image-text, and text-only data is crucial for achieving state-of-the-art (SOTA) few-shot results across multiple benchmarks, compared to other published pre-training results. Further, we show that the image encoder together with image resolution and the image token count has substantial impact, while the vision-language connector design is of comparatively negligible importance. By scaling up the presented recipe, we build MM1, a family of multimodal models up to 30B parameters, consisting of both dense models and mixture-of-experts (MoE) variants, that are SOTA in pre-training metrics and achieve competitive performance after supervised fine-tuning on a range of established multimodal benchmarks. Thanks to large-scale pre-training, MM1 enjoys appealing properties such as enhanced in-context learning, and multi-image reasoning, enabling few-shot chain-of-thought prompting.

A practical guide to constructing and retrieving information from knowledge graphs in RAG applications with Neo4j and LangChain

Graph retrieval augmented generation (Graph RAG) is gaining momentum and emerging as a powerful addition to traditional vector search retrieval methods. This approach leverages the structured nature of graph databases, which organize data as nodes and relationships, to enhance the depth and contextuality of retrieved information.

Redefining internet surfing by transforming natural language instructions into seamless browser interactions.

This is an upcoming high performance computing book titled “Algorithms for Modern Hardware” by Sergey Slotin.

Its intended audience is everyone from performance engineers and practical algorithm researchers to undergraduate computer science students who have just finished an advanced algorithms course and want to learn more practical ways to speed up a program than by going from O(nlog⁡n)O(nlogn) to O(nlog⁡log⁡n)O(nloglogn).

A low-code way to learn AI - Learn how AI works from a real LLM implemented entirely in Excel

• Marking a major investment in Meta’s AI future, we are announcing two 24k GPU clusters. We are sharing details on the hardware, network, storage, design, performance, and software that help us extract high throughput and reliability for various AI workloads. We use this cluster design for Llama 3 training.
• We are strongly committed to open compute and open source. We built these clusters on top of Grand Teton, OpenRack, and PyTorch and continue to push open innovation across the industry.
• This announcement is one step in our ambitious infrastructure roadmap. By the end of 2024, we’re aiming to continue to grow our infrastructure build-out that will include 350,000 NVIDIA H100 GPUs as part of a portfolio that will feature compute power equivalent to nearly 600,000 H100s.

Transformer Debugger (TDB) is a tool developed by OpenAI's Superalignment team with the goal of supporting investigations into specific behaviors of small language models. The tool combines automated interpretability techniques with sparse autoencoders.

TDB enables rapid exploration before needing to write code, with the ability to intervene in the forward pass and see how it affects a particular behavior. It can be used to answer questions like, "Why does the model output token A instead of token B for this prompt?" or "Why does attention head H attend to token T for this prompt?" It does so by identifying specific components (neurons, attention heads, autoencoder latents) that contribute to the behavior, showing automatically generated explanations of what causes those components to activate most strongly, and tracing connections between components to help discover circuits.

Here, we'll cover the derivations from scratch to provide a rigorous understanding of the core ideas behind diffusion. What assumptions are we making? What properties arise as a result?

A reference [codebase] is written from scratch, which provides minimalist re-production of the MNIST example below. It clocks in at under 500 lines of code.

Each page takes up to an hour to read thoroughly. Approximately a lecture each.

This tutorial aims to introduce diffusion models from an optimization perspective as introduced in our paper (joint work with Frank Permenter). It will go over both theory and code, using the theory to explain how to implement diffusion models from scratch. By the end of the tutorial, you will learn how to implement training and sampling code for a toy dataset, which will also work for larger datasets and models.

We introduce a new algorithm called the Free-pipeline Fast Inner Product (FFIP) and its hardware architecture that improve an under-explored fast inner-product algorithm (FIP) proposed by Winograd in 1968. Unlike the unrelated Winograd minimal filtering algorithms for convolutional layers, FIP is applicable to all machine learning (ML) model layers that can mainly decompose to matrix multiplication, including fully-connected, convolutional, recurrent, and attention/transformer layers. We implement FIP for the first time in an ML accelerator then present our FFIP algorithm and generalized architecture which inherently improve FIP's clock frequency and, as a consequence, throughput for a similar hardware cost. Finally, we contribute ML-specific optimizations for the FIP and FFIP algorithms and architectures. We show that FFIP can be seamlessly incorporated into traditional fixed-point systolic array ML accelerators to achieve the same throughput with half the number of multiply-accumulate (MAC) units, or it can double the maximum systolic array size that can fit onto devices with a fixed hardware budget. Our FFIP implementation for non-sparse ML models with 8 to 16-bit fixed-point inputs achieves higher throughput and compute efficiency than the best-in-class prior solutions on the same type of compute platform.

GitHub

This repository contains JAX example code for loading and running the Grok-1 open-weights model.

Ollama now supports AMD graphics cards in preview on Windows and Linux. All the features of Ollama can now be accelerated by AMD graphics cards on Ollama for Linux and Windows.

Friends and colleagues have been asking me to share my perspective on the Apple Vision Pro as a product.

This started as blog post and became an essay before too long, so I’ve structured my writing in multiple sections each with a clear lead to make it a bit easier to digest — peppered with my own ‘takes’. I’ve tried to stick to original thoughts for the most part and link to what others have said where applicable.

Some of the topics I touch on:

• Why I believe Vision Pro may be an over-engineered “devkit”
• The genius & audacity behind some of Apple’s hardware decisions
• Gaze & pinch is an incredible UI superpower and major industry ah-ha moment
• Why the Vision Pro software/content story is so dull and unimaginative
• Why most people won’t use Vision Pro for watching TV/movies
• Apple’s bet in immersive video is a total game-changer for Live Sports
• Why I returned my Vision Pro… and my Top 10 wishlist to reconsider
• Apple’s VR debut is the best thing that ever happened to Oculus/Meta
• My unsolicited product advice to Meta for Quest Pro 2 and beyond

Bluesky was created to put users and communities in control of their social spaces online. The first generation of social media platforms connected the world, but ended up consolidating power in the hands of a few corporations and their leaders. Our online experience doesn’t have to depend on billionaires unilaterally making decisions over what we see. On an open social network like Bluesky, you can shape your experience for yourself.

Today, we’re excited to announce that we’re open-sourcing Ozone, our collaborative moderation tool. With Ozone, individuals and teams can work together to review and label content across the network. Later this week, we’re opening up the ability for you to run your own independent moderation services, seamlessly integrated into the Bluesky app. This means that you'll be able to create and subscribe to additional moderation services on top of what Bluesky requires, giving you unprecedented control over your social media experience.