By Julia Husainzada, Data Science Student and Founder & President of the Responsible Computing Club, San José State University
What industry engineers taught us about production-ready AI at Open Source AI: Building Reliable AI Systems, San José State University, October 19, 2026.
Building an AI demo has never been easier. A few prompts, an API key and a free afternoon are enough to get something impressive running on a laptop. The next challenge: turning that demo into a scalable system customers can trust.
That gap was the focus of Open Source AI: Building Reliable AI Systems, an evening event at San José State University drawing students and developers. The event was hosted and organized by Susan Malaika (IBM, Responsible AI at LF AI & Data), in collaboration with SJSU’s Career Center and Responsible Computing Club. Jennifer Hernandez, IBM’s Client, University & Community Engagement Leader, helped organize the event and connected with students about opportunities at IBM throughout the evening.
Across five sessions, speakers shared tools that make AI more reliable and how students can start contributing today.
Four cutting-edge tools for reliable AI
Each tool tackles a different part of taking AI from demo to deployable: making LLM calls reliable, keeping AI safe, building it securely and grounding it in reliable data.
Mellea: LLMs as building blocks
Basel Shbita opened with a question: how do we get from “agentic demo soup” to robust software? He described a familiar scenario: a developer vibe-coded a demo on a laptop, the manager loved it, and then it had to become software the whole company could ship to customers.
Basel explained that many of today’s agents are glorified prompts. They grow over time, break easily and are often tuned to one specific model. An instruction like “Do not hallucinate. My career depends on this.” does not make a system secure, and essay-length prompts are expensive to run at scale.
To move beyond fragile prompts, Basel introduced Mellea, a Python library that sits between the application and the model, treating the LLM as a building block rather than the whole system. A task is broken into smaller steps, each with its own instructions, requirements and sampling strategy. Every call follows an instruct, validate, repair pattern and must return the data type the developer defined. The Mellea team calls this approach generative computing, which offers several benefits over prompt engineering:
- Small models, big results. Well-structured steps let small open-weight models do big-model work at a fraction of the cost.
- Flexibility. Models and inference engines can be swapped with little effort.
- Control. Behavior is defined by the developer’s program, not by a fragile prompt.
Mellea is open source, so students can install it with pip install mellea and contribute to the project.
ASAGO: From policy to production
Earning customers’ trust means turning safety principles into practices that can be tested at scale.
Muneeza Azmat, a Senior ML Engineer on Red Hat’s AI safety team, shared the three questions every organization must answer to deploy AI safely:
- What should our AI safety look like? This is usually defined by legal and governance teams in policy documents.
- What are the actual technical boundaries of our system? Engineers find these through evaluations and red teaming, using metrics such as attack success rate.
- Do the two line up?
The answers come from very different teams. A policy might say an AI system must not act “without appropriate authorization,” but testing that rule looks completely different for an agent that sends email than for one that edits a database. This is exactly where automation can help engineers the most.
ASAGO is an open source project that automates the pipeline from policy to production. Given a policy document, a use case and an AI system, ASAGO:
- Treats the policy document as the ground truth, the single agreed source every test is measured against
- Maps sections of the policy to established AI risk taxonomies
- Generates risk scenarios based on the tools the agent can access and the actions they allow
- Translates those scenarios into tests for open source red-teaming tools such as Garak, with rubrics that decide whether each attack succeeded
Because every test traces back to the policy, compliance teams get traceability and a measure of how much of the policy is covered. ASAGO is built for everyone in the pipeline: AI engineers get concrete tests to run, and compliance and risk officers get evidence that the system follows policy. As a community-governed open source project, ASAGO welcomes new contributors.
Bob: The developer as conductor
Building AI securely matters just as much as testing it, especially as AI tools write more of the code.
As James Busche put it, developers are shifting from writing every line of code to conducting the orchestra. He showed how Bob, IBM’s AI development environment, builds security into that process. Bob acts only with the permissions the developer grants, and it analyzes the project in front of it, picks the tools it needs and iterates with the developer.
Bob builds reliability into every stage of development:
- Security from the start: embedding policy enforcement and continuous security validation instead of saving security for the end
- Rules: applying global or project rules, such as requiring test cases for all new code
- Onboarding: explaining an unfamiliar codebase, or checking a developer’s understanding of a GitHub issue before work starts
- Documentation: generating and augmenting codebase documentation
Bob offers three modes (Ask to explain, Plan to map out a project, Agent to build) and selects the right model for each task, starting with small, inexpensive models and stepping up only when needed. James also covered MCP, which “gives AI arms” to reach live data such as databases, and skills, reusable instructions for tasks done often. His advice on prompting: be specific and provide context up front.
Docling: Reliable AI starts with reliable data
Even the best-built AI system is only as good as its data.
Ming Zhao reminded the audience that data is still at the core of reliable AI, and that small errors in processing can become large errors in meaning. A document’s real value lives in its tables, headings and reading order, along with its words.
Docling, an open source project started by IBM Research and now hosted by the LF AI & Data Foundation, converts common formats such as PDFs, Excel files, HTML, images and audio into AI-ready formats such as JSON. It is lightweight, runs locally and comes with a simple CLI. Highlights include:
- Layout and table recognition that preserves document structure
- Visual grounding for RAG: chunks keep metadata and bounding boxes so a RAG system can point to exactly where in a document an answer came from
- A small vision model (about 258M parameters, with over 950K downloads on Hugging Face) that processes a page in about 4 seconds
- Chart-to-CSV, formula recognition, code detection and multilingual support
- Structure-aware chunking: rather than splitting a paragraph in half, Docling splits by section and attaches each section’s heading as context
- Cost efficiency: a new AI-native document format used about 40% fewer tokens than HTML on a sample table, and Docling can be around 50x more cost-effective than running every page through a large vision model
Anyone can explore Docling through the Hugging Face demo space, work through the prebuilt workshops in the repository, or add Docling as an MCP server to an AI assistant.
Advice for students
Together, these tools show what it takes to build reliable AI, and the speakers shared how students can start building those skills:
Basel Shbita (Mellea): Divide and conquer. Break problems into subproblems. That is the whole practice of engineering and is exactly how Mellea approaches LLMs.
Muneeza Azmat (ASAGO): Get involved. No single organization has all the answers when it comes to making AI safe and secure at scale. ASAGO holds biweekly meetings and has Slack channels where anyone can share input or contribute.
James Busche (Bob): Start experimenting for free. Open source LLMs can run on a personal laptop, a free and inexpensive way to start building with AI today.
Ming Zhao (Docling): Focus on validation. Validation systems are likely to become one of the largest sectors in AI over the next few years. Many organizations adopted AI without properly validating its work, so both input data and outputs need significant validation.
Susan Malaika (Host): Get practical experience. Join a group or an open source project, take an internship or build something real. Companies are looking for practical experience.
Jennifer Hernandez (Host): You don’t have to know everything. Demonstrate that you’re willing to learn, experiment, ask good questions, and work through challenges. Especially in AI, tools will keep changing. The ability to learn and adapt is a skill you can carry throughout your career.
How to get involved in LF AI & Data
Arnaud Le Hors closed the talks by showing students how to get involved, starting with one key point: you do not need to be a member of the LF AI & Data Foundation to participate in its projects. All it takes is showing up. His advice: don’t be shy, because open source communities welcome people of all skills and backgrounds.
The LF AI & Data Foundation is an open source organization under the Linux Foundation umbrella. There are two main ways to take part: joining the discussions and documents produced by working groups, or contributing to projects. Good first steps include:
- Check the community calendar. Arnaud called it the key to everything. Working groups hold regular, open calls on Zoom, each with its own focus.
- Contribute to a project. Browse the projects, then head to a project’s GitHub and Slack. Fixing documentation or building a demo is a great first start, and students are already making a real impact: many of ASAGO’s pull requests come from students at North Carolina State University.
- Sit in on the Technical Advisory Council (TAC). The TAC oversees every project and working group in the foundation. Its meetings are open and public, which makes them an easy way to see what is happening and hear updates from projects.
- Join the conversation and the events. The LF AI & Data Slack is open to all, and the events page lists upcoming conferences and meetups.
To start with the tools from this event, explore Docling on GitHub, join ASAGO‘s community meetings, try Mellea or check out Bob.
Students are already making waves
Inspired by the talks, our team from the Responsible Computing Club, Julia Husainzada, Surabhi Bage, Tehkhum Sultanali, Yuwen Zhang, and Pouya Anvari, took three of the tools into the IBM Responsible AI track at the Sparthill Hackathon and won Best Overall Project.
We built ClearStand, an agentic Responsible AI auditing platform. Developers submit a GitHub repository, and ClearStand evaluates whether the AI application is ethical, transparent and inclusive, then proposes code fixes for a human to approve.
ClearStand’s home page: paste a GitHub repo and get a Responsible AI report card covering ethics, transparency and inclusion.
What’s Next
Student engagement with open source AI at SJSU is growing, and events like this one open doors to mentorship, internships and impactful contributions. Taking AI from demo to deployable takes reliable LLM calls, safety testing, secure development and trustworthy data, and students can start practicing every one of them today.
So, what should you do next? Join a community call, pick a project and start building!
Learn more about the LF AI & Data Foundation and its projects at lfaidata.foundation.
Thank you to Susan Malaika, Jennifer Hernandez, the speakers, the SJSU Career Center, the Responsible Computing Club and everyone who attended.
