Using LLMs for Semantic Content Generation

Tara Behrend and I have just published these fully open-access guidelines, plus accompanying code for Qualtrics, to use LLMs/AI to create custom content for surveys and experiments, quantitative or qualitative! The code for Qualtrics is as close to plug-and-play as we could make it, only requiring one copy-paste followed by changing a few settings at the top of the code block. It enables researchers to easily: 1) Create unique AI-generated content per participant (Case 2) 2) Engage participants in an LLM-based conversation with a researcher-designed system prompt (Case 4) 3) Experimentally assign participants to different LLM configurations (Case 5) My hope is that this tool increases access to LLMs for social scientists of all backgrounds. All you need is a Qualtrics account (provided for free by many universities) and a OpenAI API key. Research studies with a few hundred participants will generally cost less than $5 in API credits from OpenAI. Beyond the software itself, we developed a framework for the general use of LLMs to create content for research participants to experience/react to: Case 1) LLM as Research Assistant Case 2) LLM as Adaptive Content Provider Case 3) LLM as External Resource Case 4) LLM as Conversation Partner Case 5) LLM as Research Confederate Across cases, we provided detailed instructions on how to effectively engineer an LLM for research, including an iterative design thinking framework for prompt engineering and foundation model specification, as well as recommendations for a comprehensive audit before launch. We also present a nine-dimensional model of prompt design alongside recommendations for how to create effective prompts for research! I hope you find it useful, and I'm happy to help troubleshoot as you explore it! https://lnkd.in/gwtfH-HG

Split Smarter, Not Random: The Semantic Chunking Guide. 📚💡 Most RAG systems fail before they begin. They used outdated chunking methods that: ✂️ Slice texts by characters count 🚸 Break paragraphs without regard for meaning Imagine reading a book where someone randomly tore pages in half. That's what traditional chunking does to your data. Semantic chunking is a smarter approach that follows meaning. Let's breaks down the main approaches: 1️⃣ Embedding-Similarity Based Chunking ▪️ The system determines where to break text by comparing the similarity between consecutive sentences. ▪️ Using a sliding window approach, it calculates the cosine similarity of sentence embeddings. ▪️ If the similarity drops below a set threshold, the system identifies a semantic shift and marks the point to split the chunk. Like listening to a playlist: you can tell when one song ends and another begins. Embedding Chunking spots those natural transitions between ideas. 2️⃣ Hierarchical-Clustering Based Chunking ▪️ The system analyzes relationships between all sentences at once, not just neighbors. It starts by measuring how similar each sentence is to every other sentence in the text. ▪️ These similarities create a hierarchy—like a family tree of ideas. When sentences show strong similarity, they cluster together into small groups. ▪️ These small groups then merge into larger ones based on how closely they relate. Like organizing a library: books get grouped by topic, then broader categories, until you have a natural organization that makes sense. 3️⃣ LLM-Based Chunking This newest approach uses LLMs to chunk text based on semantic understanding. ▪️ The first step is to feed the text to an LLM with specific chunking instructions. ▪️ The LLM then identifies key ideas and how they connect, rather than just measuring similarity. ▪️ When it spots a complete thought or concept, it groups these propositions into coherent chunks. Imagine having a skilled editor who knows exactly where to break your text for maximum clarity. ⚙️ Which method will produce optimal outcomes depends on your use case: ▪️ Want precision? Go with LLM-Chunking ▪️ Want speed? Go with Embedding-Similarity ▪️ Need to preserve relationships? Go with Hierarchical-Clustering Ready to implement? Get the full technical breakdown👇

Here’s how I build an LLM content strategy in 2025: — 𝗦𝘁𝗲𝗽 𝟭: 𝗥𝘂𝗻 𝗮𝗻 𝗔𝗜 𝗩𝗶𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 𝗔𝘂𝗱𝗶𝘁 → Where is your brand mentioned in ChatGPT? → What prompts surface your content? → Where are competitors cited that you aren’t? This shows you the gap—and the opportunity. 𝗦𝘁𝗲𝗽 𝟮: 𝗘𝘅𝘁𝗿𝗮𝗰𝘁 𝗕𝘂𝘆𝗲𝗿-𝗟𝗲𝗱 𝗣𝗿𝗼𝗺𝗽𝘁𝘀 → Use Google PAA, Reddit, and call transcripts → Identify bottom-of-funnel prompts → Reverse-engineer the decision journey Don't guess what users ask LLMs. 𝘍𝘪𝘯𝘥 𝘰𝘶𝘵. 𝗦𝘁𝗲𝗽 𝟯: 𝗜𝗱𝗲𝗻𝘁𝗶𝗳𝘆 𝗬𝗼𝘂𝗿 “𝗙𝗨𝗤𝘀” → Frequently Unasked Questions = massive opportunity → These are the gaps no one is filling → Original research helps you 𝘣𝘦 the source New answers → new citations. 𝗦𝘁𝗲𝗽 𝟰: 𝗕𝘂𝗶𝗹𝗱 𝗮 𝗦𝗲𝗺𝗮𝗻𝘁𝗶𝗰 𝗔𝗻𝗰𝗵𝗼𝗿 → Create a single “entity home” page → Add schema + “sameAs” links → Include clear founding, product, customer, and differentiator info LLMs need to know who you 𝘢𝘳𝘦. This is how they learn. 𝗦𝘁𝗲𝗽 𝟱: 𝗖𝗼𝗿𝗿𝗲𝗰𝘁 𝗬𝗼𝘂𝗿 𝗠𝗲𝘀𝘀𝗮𝗴𝗶𝗻𝗴 𝗘𝘃𝗲𝗿𝘆𝘄𝗵𝗲𝗿𝗲 → Align bios, PR, press pages, affiliate blurbs → Keep descriptions consistent → Don’t confuse the model If your brand presence is inconsistent, the model won’t know who you are. 𝗦𝘁𝗲𝗽 𝟲: 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗲 𝗳𝗼𝗿 𝗔𝗜-𝗣𝗮𝗿𝘀𝗮𝗯𝗹𝗲 𝗖𝗼𝗻𝘁𝗲𝗻𝘁 → Short sentences. Ordered sections. → Semantic triples (subject → object → predicate) → Clear headers + natural language Think “easy to quote” — that’s the bar. 𝗦𝘁𝗲𝗽 𝟳: 𝗣𝗶𝘁𝗰𝗵 𝗖𝗶𝘁𝗲𝗱 𝗦𝗼𝘂𝗿𝗰𝗲𝘀 → Identify the blogs, pages, and publishers ChatGPT already cites → Get featured or collaborate → Build citations like you used to build links In the LLM era, citations 𝘢𝘳𝘦 authority. 𝗦𝘁𝗲𝗽 𝟴: 𝗧𝗿𝗮𝗰𝗸 𝗧𝗵𝗲 𝗥𝗶𝗴𝗵𝘁 𝗠𝗲𝘁𝗿𝗶𝗰𝘀 → Count your share of citations per prompt → Identify funnel stage breakdown (TOFU/MOFU/BOFU) → Measure growth in agentic bot activity (e.g. ChatGPT-user) No GSC for ChatGPT means you build your own dashboards. 𝗦𝘁𝗲𝗽 𝟵: 𝗧𝗿𝗲𝗮𝘁 𝗬𝗼𝘂𝗿 𝗟𝗟𝗠 𝗖𝗼𝗻𝘁𝗲𝗻𝘁 𝗦𝘁𝗿𝗮𝘁𝗲𝗴𝘆 𝗟𝗶𝗸𝗲 𝗮 𝗚𝗿𝗼𝘄𝘁𝗵 𝗟𝗼𝗼𝗽 → Audit → Act → Re-measure → Repeat This is LLM-led growth. And it works. _ 📄 Save for future reference. ♻️ REPOST so others can learn too. P.S. Hit the 🔔 for weekly #AI + #SEO updates.

Day 16/30 of LLMs/SLMs - Retrieval-Augmented Generation (RAG) Large Language Models are powerful, but they have a fixed memory. They cannot know anything that happened after their training cut-off, and they struggle with facts that were never part of their dataset. When they lack the right information, they guess. The result is fluent but unreliable text — the hallmark of hallucination. Retrieval-Augmented Generation (RAG) fixes that by giving models a way to look up information before they answer. RAG is best understood as a three-stage pipeline, and LangChain has become the de facto standard framework for building each stage efficiently. 𝐈𝐧𝐠𝐞𝐬𝐭𝐢𝐨𝐧 𝐚𝐧𝐝 𝐈𝐧𝐝𝐞𝐱𝐢𝐧𝐠 You start by collecting and preparing your documents. LangChain’s loaders handle PDFs, web pages, CSVs, and APIs. These documents are then split into smaller, semantically meaningful chunks and converted into embeddings using models like OpenAI’s text-embedding-3-small, SentenceTransformers, or InstructorXL. Those embeddings are stored in a vector database such as FAISS, Pinecone, Weaviate, or Chroma, which lets you perform similarity search later. 𝐑𝐞𝐭𝐫𝐢𝐞𝐯𝐚𝐥 When a query arrives, LangChain converts it into an embedding and searches the vector store for the most relevant documents. Retrieval strategies vary — basic similarity search, maximal marginal relevance (MMR) to diversify context, or hybrid retrieval that mixes semantic and keyword search. The retrieved text chunks are then added to the prompt as contextual grounding. 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐨𝐧 The LLM receives the augmented prompt containing both the user query and retrieved passages. It synthesizes an answer based on that external knowledge. LangChain manages prompt templates, context formatting, and memory across queries, making the process modular and repeatable. 𝐖𝐡𝐲 𝐑𝐀𝐆 𝐌𝐚𝐭𝐭𝐞𝐫𝐬 RAG fundamentally improves factual accuracy and trust. On benchmarks such as Natural Questions and TriviaQA, a base model like LLaMA 2-13B might achieve 45 F1, while RAG-augmented versions reach 65–70 F1 - matching much larger and costlier models. 𝐆𝐞𝐭𝐭𝐢𝐧𝐠 𝐒𝐭𝐚𝐫𝐭𝐞𝐝 𝐰𝐢𝐭𝐡 𝐋𝐚𝐧𝐠𝐂𝐡𝐚𝐢𝐧 𝐑𝐀𝐆 If you want to experiment, LangChain makes it approachable. A minimal prototype takes fewer than 20 lines of code. Here’s a good progression 👉 Start with the LangChain tutorial: https://lnkd.in/gUpHpkKT 👉 Add a vector store: Try Chroma for local experiments or Pinecone for scalable hosting. 👉 Experiment with retrieval methods: compare similarity search vs. MMR. 👉 Integrate your own data: ingest PDFs, database exports, or web content. 👉 Deploy a chain: connect your retriever, model, and prompt template into a single workflow. Tune in tomorrow for more SLM/LLMs deep dives. -- 🚶➡️ To learn more about LLMs/SLMs, follow me - Karun! ♻️ Share so others can learn, and you can build your LinkedIn presence!

🧠 #LLMs Think in Vectors, Not Keywords I like to envision LLM's latent space (brain) as a massive fractal cloud. You can zoom in to any part of it and find structure. Zoom one place and you are in the Soccer Cleats purchase intent zone of the latent space, shift slightly to the right and maybe you are now in Soccer Ball information intent zone of the latent space. What the LLM knows about your brand is also a fractal cloud, situated inside the LLM, and overlapping with everything known about your brand. Your goal should be to create a fractal cloud that matches the fractal cloud shapes of the high intent latent spaces related to your product or service. Traditional #SEO was about keyword matching. LLMs now answer based on entity salience, coherence, and fit-to-intent vector. So the question becomes: What latent concept clusters does my brand inhabit, and where am I “closer” or “further” than my competitors? 🔭  New Brand Positioning = Optimal Semantic Distance “Optimally distanced” means: Not too broad: Generic brands are skipped in favor of those that offer high signal for a specific intent vector. Not too niche: If your brand is semantically distant from common queries, you’ll never be surfaced—even if you’re the best at what you do. So the strategy becomes: Map the latent semantic space of need-states your customers express. Then triangulate where your brand can “anchor” itself to be the most probable answer vector across multiple agentic inferences. 🧩  Tactics for Agentic Brand Optimization Semantic Mapping of Intent Graphs Use LLMs to simulate thousands of user questions in your category. Embed them and cluster to find need-states. Map where your brand is mentioned, and where it should be. Coherence Seeding - Place your brand in high-coherence contexts: - Longform thought leadership - Expert co-citation (partner pages, academic links) - LLM-training-friendly docs (structured, well-labeled) - Canonicalization Hacking Ensure your brand is defined as a canonical fit. - “X is known for…” pages - Descriptive metadata in schema.org / structured data - Embedding yourself into toolkits, glossaries, walkthroughs used by RAG systems - Distilled Value Propositions Agents prefer clarity. Use highly distilled one-liners: - “The fastest no-code API generator for startups” - “Trusted by NASA to visualize quantum simulations” Interaction Instrumentation Feedback loops matter: - Build tools, calculators, visualizers—things that get invoked by LLMs Let their success become your reinforcement signal 🔮 Bonus: Think in Agent Choreography Agents don’t just answer questions—they invoke chains of tools. Be the tool. Or be the source cited by the tool. Or be the brand evoked in the process as the best example of coherence.