Agentic Large Language Models for Conceptual Systems Engineering and Design

Published:

Agentic Large Language Models for Conceptual Systems Engineering and Design

Agentic LLMs for Conceptual Systems Engineering and Design introduces a novel multi‑agent architecture that leverages large language models to automate requirements extraction, functional decomposition, and physics‑model code generation via a JSON‑serializable Design‑State Graph (DSG). Through a nine‑role MAS and a two‑agent baseline (2AS), the paper evaluates 60 experiments using both Llama 3.3 70B and a reasoning‑distilled DeepSeek R1 70B, reporting on JSON validity, requirement coverage, embodiment tagging, code executability, workflow completion, runtime, and graph granularity.

Key Features

  • Design‑State Graph (DSG): Typed, directed multigraph bundling requirements, physical embodiments, and Python‑based physics models into discrete nodes for tool‑interop and version control
  • Multi‑Agent System: Nine specialized agents (Extractor, Supervisor, Generator, Coder, Reflector, Ranker, Meta‑Reviewer, Orchestrator, Worker) orchestrating iterative design processes with structured JSON outputs
  • Two‑Agent Baseline (2AS): Simplified Generator‑Reflector loop serving as an ablation to assess the benefit of hierarchical orchestration
  • Empirical Study: 60 runs across two LLMs (vanilla vs reasoning‑distilled), three sampling temperatures (0.0, 0.5, 1.0), and two system architectures, measuring seven metrics from JSON validity to graph size

Technical Stack

  • LLMs: Llama 3.3 70B & DeepSeek R1 70B
  • Orchestration: LangChain & LangGraph for managing agent workflows
  • Data Modeling: Pydantic for DSG schema, JSON for state serialization
  • Execution: Python scripts validated via subprocess; regex parsing for metric extraction
  • Tools: Python REPL, structured function‑calling for tool integration

Findings

  • MAS yields finer‑grained DSGs (≈ 5–6 nodes) but lower average code executability compared to 2AS
  • DeepSeek R1 boosts completion rates, though requirement coverage remains below 20%
  • Structured JSON outputs enable tool interoperability and version control
  • Hierarchical orchestration provides more detailed design decomposition

Agentic Large Language Models for Conceptual Systems Engineering and Design (preprint)

This work demonstrates how agentic LLM frameworks can structure early‑stage systems design, while highlighting open challenges in requirement fidelity and physics‑correct code generation.