Cybersecurity Platform

In March 2023, I joined Pinochle AI as Chief Engineer at the invitation of its CISO, Dr. Kaushik Hatti, and the CEO, Mr. Ranghan Venkatraman. Our objective was to provide comprehensive cybersecurity for large enterprise IT networks in a fully automated, intelligent way. This was becoming the need of the hour then (and even more so now) for large enterprises as the cyber adversaries were becoming more potent with usage of AI tools. We set out to build a cybersecurity platform that can, at first, detect the vulnerabilities and misconfigurations in large IT networks. This post details the product development story and the basic architecture behind it.

I was given the opportunity to start with an energetic young team of software engineers and a clean slate to achieve our goals. The company had an active service arm that provided cybersecurity services for enterprises but the challenges were increasing as the scale, complexity and the strength of adversaries grew. At first, our idea was to fully automate the data collection and curation in a robust way. We began with scanning and monitoring the external attack surface of the network - and this would become the first version of our product.

The Scanner Engine

After a few weeks of deliberations and pilot projects, I arrived at the following tech stack for the platform:

• A React/TypeScript frontend dashboard for users to sign-in and request for security scans of their public facing domains and subdomains (e.g. www.example.com)
• FastAPI/Python backend handles secure user authentication, provides the REST API endpoints for the dashboard, and generates a PDF security report of the scans.
• MongoDB serves as the primary database to store application data
• Scanner Engine is the heart of the platform - this is a Python FastAPI orchestrator which uses celery heavily for task scheduling. The engine is a combination of various proprietary security tools, functions and some open-source libraries.

I kept running into the problem of synchronizing various people working on different application components of the platform. I needed a robust solution that will make sure everyone stays on the same page regarding the schema of the security data we are collecting at large scales - partcularly as the schema was fast evolving due to rapid development of features and tools in the scanner engine.

I designed a Python module that will act as our database-schema and data-model tying together every application component's functionality. This ScanModel was built using various classes all built-on the wonderful Pydantic library and provided type safety, data validation and serialization. I used this python module as a git submodule in many application components and setup automations (using Gitea Actions) to auto-update dependent repos on changes.

I prioritized full automation of DevSecOps - we used self-hosted Gitea, Gitea Actions for CI/CD, Docker and Docker Compose for deployment, AWS for the cloud infrastructure. I ensured the application is fully secure, contained layers of security protections and the development followed best security practices. At every point of development, our Red Team helped us to make sure there are no security gaps in our application.

How to get insights from Data

Our first version of the product with the Scanner Engine provided useful data about the attack surface and we were able to assign some scoring metrics to identify which configuration needs to be fixed/improved. But as far the security of an enterprise IT network is concerned, this is not even scratching the surface.

This was the time LLMs and RAG were hot in the news. We did setup some RAG pipelines to helpout with some user chat queries about their security scan data. Hallucinations have improved in LLMs since then, but we could not allow any inaccuracy in our predictions/insights as cybersecurity is not to be taken lightly at any cost. We needed a method that will ground the insights on facts of the data alone. At this point, I deliberated with Dr. Kaushik about various approaches and we finally turned to Knowledge Graphs and Semantic Technologies to anchor our platform.^[1]

The Graph Engine

Knowledge Graph (KG) is a graph that is built with context/meaning as part of the data model. The context/meaning is encoded in an ontology and the data model is called RDF. In simple terms, we want to encode our understanding of the cybersecurity domain in an ontology (think of it like a dictionary of words for a language) and store our data in a graph network consisting of triples using the terms of the ontology (think of it like writing a paragraph of meaningful sentences in that language). KGs come with lots of advantages like conveying context, graph algorithms and traversal, etc. but the most important aspect is that they can anchor the context for LLMs. The results from the graph can also be logically verfied using a reasoning engine.

I set out to define a custom ontology for cybersecurity domain. This must include all aspects of the domain: IT infrastructure, software, hardware, networking concepts, standard security and compliance frameworks (NIST, CIS, MITRE ATT&CK, D3FEND, CVE, etc.). While the semantic technology ecosystem was heavily java-based, I wanted to keep our entire stack in Python as we could take advantage of advancements in LLMs and their tooling faster. After a few weeks, I designed the following architecture for the platform:

• Ontology Builder: This is a Python module which uses the excellent Owlready2 library for defining the ontology
  • We developed many smaller ontologies describing specific parts of the domain and brought them together
• Knowledge Graph Builder: An Apache Airflow service that ingests the incoming data, adapts it to fit within the ontology and stores it in our Graph Database
• Apache Jena Fuseki is our main graph database (RDF Store) - supports SPARQL queries
• A FastAPI service that can directly convert some user queries into appropriate SPARQL queries with the aid of LLM Agents and Function calling.

The KG Builder also has some agents running alongside to identify security vulnerabilities from the KG and enrich the graph data with more insights.

As we are dealing with client-side sensitive information mainly, we had to rely on our own self-hosted open-source LLMs for all the work. This constraint has been a blessing for me personally as I have learned a great deal about controlling LLM behaviour, reducing hallucinations, data grounding and sovereign AI at enterprise scale. In my opinion, integrating private AI models closely with business data, improving efficiency and getting actionable insights is the future.

Footnote