Practical Prompting Techniques and AI Scientific Breakthroughs

TODAY'S HIGHLIGHTS:

• AI is becoming the "super secret answer machine" capable of seeing patterns across wide and deep data fields that humans cannot comprehend

• Practical prompting techniques from experts, including using Markdown instead of PDFs and structuring data for optimal LLM performance

• Security concerns with MCPs (Model-Calling Protocols) and best practices for safe implementation

• The power of creating detailed briefs before coding with AI agents for better outcomes

INTRODUCTION:

Welcome to the AI Biz Hour, where Andy Wergedal (@andywergedal) and John Allen (@AiJohnAllen) explore the cutting edge of AI business innovation. In today's packed episode, the discussion ranged from philosophical reflections on AI's capabilities to highly technical advice on prompting techniques, AI agent development, and securing MCP implementations. Industry experts shared invaluable insights about how businesses can leverage AI tools effectively while avoiding common pitfalls.

MAIN INSIGHTS:

AI as the "Super Secret Answer Machine"

Andy kicked off the discussion highlighting how AI is becoming the ultimate answer machine across domains like physics, chemistry, neural networks, and medicine. "It's going to be able to see patterns that we don't see yet, because it has access to such a wide, thin, but also wide and deep data fields," Andy noted. While humans retain intuition that AI lacks, AI's computational power is transforming fields from space trajectory calculations to legal precedent analysis. The value of certain specialized knowledge is changing as AI democratizes access to information that once required expensive expert consultation.

Prompting Best Practices from Industry Experts

Stunspot, a professional prompt engineer who pays rent by writing prompts, shared insights on AI's capabilities. "In the same way that computers are industrialized logic and reason, LLMs are industrialized intuition. They don't run on rules—they run on vibes," he explained, challenging the notion that AI lacks intuition. Stunspot emphasized that understanding AI's foundation in human meaning is crucial for effective prompting.

Umesh provided exceptional technical advice about working with LLMs at a deeper level:

• Use Markdown instead of PDFs when providing context to models: "I've found over a period of time that markdowns perform better with these models"

• Structure data in a human-readable, less-nested format (like XML) instead of deeply nested JSON for better processing

• Create detailed briefs before coding with AI agents: "Spend more time on creating the document rather than working on the code itself"

The Power of Meta-Prompts and Knowledge Libraries

Several speakers shared techniques for optimizing AI interactions:

Umesh described his system of meta-prompts: "Basically, it's a simple items code that gets automatically updated. We call it a knowledge graph." This approach allows users to give one-line prompts that trigger comprehensive responses based on pre-defined contexts.

Andrew added practical advice for non-coders using AI tools: "In terms of giving instruction to your coding agents, —it's pretty much the default instruction file that Quart expects in every project." He recommended breaking down complex projects into multiple markdown documents with a master document that links to specific components.

Simon highlighted the value of using tables of information to assist with prompting: "Once you have all of these categories built up, you actually have basically infinite amount of prompt exploration that you can do."

Umesh's Advanced Technical Frameworks

Umesh delivered exceptional technical insights about optimizing LLM interactions through sophisticated approaches:

Meta-Prompt Systems and Knowledge Graphs

"I have a system of meta-prompts... it's a simple items code that gets automatically updated. We call it a kind of knowledge graph." This approach enables users to:

• Create domain-specific meta-prompts (like for science fiction writing)

• Give one-line prompts that trigger comprehensive responses

• Link everything into a DSPy library that evolves with use

Umesh's system runs these meta-prompts through preferred models (Anthropic, Gemini, OpenAI) and performs re-ranking to generate optimized prompts from minimal input.

Document Structure for AI Interaction

Umesh provided specific technical advice about document formatting:

"I don't use PDF, I use markdown because I've found over a period of time that markdowns perform better with these models." This seemingly small detail can significantly impact results.

For complex projects, he recommends a hierarchical approach:

• Create a master markdown document with architecture overview

• Develop separate markdown documents for specific functionalities

• Link documents together in a structured repository

• Reference specific documents when focusing on particular tasks

This approach allows for targeted updates: "If you want to make a small change, or if you learn through interaction with the model... you can just go to that particular document, make that change."

Early Direction Setting for LLMs

Umesh shared a crucial insight about LLM behavior: "The earlier the model gets drips into the direction that you want it, the better it is for you. Then it remains faithful to the context."

He explained that establishing direction at the beginning is vital as "it is very difficult to change the direction" later. This early direction setting helps the model create a coherent mental map that persists even after 30-40 turns of conversation.

XML vs. JSON for Model Communication

In a technical exchange with another participant, Umesh recommended: "Converting JSON output from MCPs into XML format before feeding it back to the LLM." This is due to a historical bias in training data towards tagged formats from Web 1.0 era.

"Even smaller models handle XML better, improving context recall and data structure understanding, leading to significantly higher accuracy (Six Sigma level in early testing)."

The Power of Structured Briefs for AI Coding

Umesh emphasized the critical importance of detailed briefs for non-coders working with AI:

"Spend literally 6 hours today to build just one simple markdown document. Which is very concise, so it's a 4-page document. But it contains everything that the model needs to know in a very, very concise manner."

This approach includes:

• Providing concise but comprehensive context

• Linking to specific APIs or best practices articles

• Instructing the model to read those resources before proceeding

• Creating a knowledge structure that guides the model effectively

"As a non-coder, I would say spend more time on creating the document rather than working on the code itself, because the better document you have, the better will be the outcome."

Sponsor JanCarlos Decision Clarity Framework

JanCarlos (@jancarlosbtc), is sponsoring the AI Biz Hour for the next three weeks. He’s introduced his powerful Decision Clarity Framework, a system designed to help individuals and businesses use AI to navigate complexity and make better, faster decisions.

JanCarlos shared that the framework was born from his personal journey with autism. Faced with challenges in communication and overwhelming sensory input from a young age, he was forced to develop meticulous systems to understand the world and make clear decisions. This personal experience became the foundation for a professional framework that anyone can use.

Security Concerns with MCPs (Model-Calling Protocols)

Wes, who recently returned from a weekend away from screens, raised important security considerations for businesses implementing AI systems, particularly Model-Calling Protocol (MCP) servers:

"When you're consuming open source MCP servers, be careful just kind of pulling these MCP servers from the ether," he warned. Even well-intentioned developers might include dependencies they haven't properly vetted, potentially exposing sensitive information.

Jason added: "I would definitely recommend normal virus scans and malware scans, maybe running stuff in a sandbox if you're that concerned."

For enhanced security, Wes recommended: "If you can, wrap it in a Docker container. You're giving your users a ton of security advantages—you're isolating the networking, you're isolating the files."

EXPERT CORNER: REAL LUCAS ON AGENT DEVELOPMENT

RealLucas shared challenges in keeping AI agents on task: "I'm now spending a lot of time trying to figure out how to keep context and how to keep agents on task." Despite explicit instructions, Lucas has experienced agents refactoring or deleting code that shouldn't be modified.

"I've had AI agents, despite absolutely explicit instructions, refactor code that's not supposed to be refactored, delete code that was not supposed to be deleted," he explained. This appears to be directly tied to the complexity of the code and dependencies, suggesting limits to current AI tools for commercial-grade applications.

QUICK HITS:

• Use Markdown instead of PDFs when providing context to AI models

• Create detailed briefs before starting AI-assisted coding projects

• Consider wrapping MCP servers in Docker containers for added security

• Break down complex projects into multiple linked documents

• Be cautious about installing MCPs from untrusted sources

RESOURCES MENTIONED:

• LM Studio: A tool for running local LLMs with MCP support

• Gemini CLI: Command line interface for Google's AI models

• Neo4j: Graph database that enhances LLM's ability to traverse complex data

• Docker: Container platform recommended for securing MCP servers

COMING UP:

Join us for tomorrow's AI Biz Hour session at 12 PM ET with a special focus on JanCarlo's Decision Clarity Framework!

CONNECT WITH AI BIZ HOUR:

• Website: aibizhour.com

• Andy: @andywergedal

• John: @AiJohnAllen

• Show: @aibizhour

CALL TO ACTION:

Don't miss out on future insights! Sign up for the newsletter at aibizhour.com to receive these valuable discussions in your inbox. The newsletter captures all the key points from each episode, making it easy to reference expert advice even if you miss the live session. Support the show by following hosts and guests, and consider buying John Allen a coffee through his profile if you've found value in these discussions.

A SPONSOR Gov Bid Mike

Gov Bid Mike

Looking to tap into the $7 trillion government contracting market? GovBidMike helps businesses secure government contracts and grants. With important AI procurement rule changes coming in October 2024, now is the time to position your business. Mention AI Biz Hour for a 10% discount on services. Government contracts increasingly specify American-made AI technologies and interoperability requirements. Visit biddata.ai to learn how to navigate the complex world of government procurement.