dotagent

> Hey there, Friend!
> This project is still in the "just for friends" stage. If you want to see what we're messing with and have some thoughts, take a look at the code. We'd love your feedback or contributions.
What is Nextpy?
Nextpy is a framework for building self-modifying software.
Key Features
🚧 Guardrails
- ***Set clear boundaries:*** Users can precisely define what the AI system can and cannot do. This safeguard ensures that the AI system remains a dynamic, self-improving system without overstepping established limits.
🏗️ Greater control with structured outputs
- ***More effective than chaining or prompting:*** The prompt engine unlocks the next level of prompt engineering, offering significantly greater control over LLMs compared to few-shot prompting or traditional chaining methods.
- ***Superpowers to prompt engineers:*** It gives full power of prompt engineering, aligning with how LLMs actually process text. This understanding enables you to precisely control the output, defining the exact response structure and instructing LLMs on how to generate responses.
🏭 Powerful prompt engine
The philosophy is to handle more processing at compile time and maintain better sessions with LLMs.
- ***Pre-compiling prompts:*** By handling basic prompt processing at compile time, unnecessary redundant LLM processing is eliminated.
- ***Session state with LLMs:*** Maintaining state with LLMs and reusing KV caches can eliminate many redundant generations and significantly speed up the process for longer and more complex prompts. *(only for open-source models)*
- ***Optimized tokens:*** The engine can transform many output tokens into prompt token batches, which are cheaper and faster. The structure of the template can dynamically guide the probabilities of subsequent tokens, ensuring alignment with the template and optimized tokenization. ****(only for open-source models)****
- ***Speculative sampling (WIP):*** You can enhance token generation speed in a large language model by using a smaller model as an assistant. The method relies on an algorithm that generates multiple tokens per transformer call using a faster draft model. This can lead to up to a 3x speedup in token generation.
🤖 Better AI Generations:
- **🧠 More Effective Than Chaining or Prompt Engineering** - Next.py aligns with LLM processing patterns, enabling precise output control and optimal model utilization.
- **💡 Optimized for Code Generation** - Regardless of the LLMs, prompts, or fine-tuning used, the underlying app framework significantly impacts the efficiency of code generation. Next.py's architecture is specifically engineered to maximize efficiency.
- **💾 Session State with LLM** - Efficiently maintain state with LLMs, leveraging KV caches to convert multiple output tokens into prompt token batches. This approach reduces redundant generations, accelerating the handling of lengthy and intricate prompts. ***(only for open-source models)***
- **🧪 Detect Syntax Errors**: Test LLM-generated code, identifying and correcting LLM hallucinations, invalid Nextpy methods, and automatically generating prompts for seamless fixes.