← All work

Green-Connect: AI-Driven Discovery of Sustainability Projects for Financing

An AI platform that combines web crawling and natural language processing to find sustainability projects online that may need financing and ranks them by relevance for review by financial experts.

Natural language processing · Web crawling · Decision support
Diagram of the Green-Connect system, showing the web crawler component that retrieves online documents and the text-mining component that scores them using the Room knowledge base and the expert-defined Benchmark.
Green-Connect: the web crawler retrieves candidate documents, and the text-mining component scores their relevance using a domain knowledge base (the Room) and an expert-defined Benchmark. From Behrooz et al., Sustainability 2023 (CC BY 4.0).

The problem

Finding promising sustainability projects to finance, particularly those led by small and medium-sized enterprises, depends on professional networks, subject-matter expertise, and manual reviews of project documents. The process is time-consuming and difficult to scale, especially when relevant information is scattered across thousands of organizations and websites.

Green-Connect automates the initial search by finding and analyzing online project documents and ranking potential financing opportunities. This allows financial experts to discover and review a broader range of projects without searching every source by hand.

Data

2,300domain documents used to train the knowledge base
460expert-defined and weighted Benchmark phrases

The knowledge base (the Room) starts from 400,000 pretrained GloVe word vectors and is then trained on more than 10,000 pages of documents about financing sustainability projects, identified together with financing professionals.

Approach

Green-Connect combines two main components: an automated web crawler that discovers potential project documents online, and a domain-specific NLP system that evaluates and ranks them.

Automated web discovery

The crawler searches online sources using combinations of three groups of expert-defined keywords:

  • Sustainability: terms related to sustainable development and environmental initiatives.
  • Project development: terms indicating proposed, ongoing, or planned projects.
  • Financing: terms related to funding needs and investment opportunities.

The crawler retrieves candidate PDF documents and shortlists only those that match all three keyword groups. These documents are then passed to the NLP system for further analysis.

Domain-specific NLP

The text-mining system is built around three elements:

  • Room: a knowledge base of word embeddings that captures how terms used in financing sustainability projects relate to each other.
  • Benchmark: a collection of 460 expert-defined and weighted phrases representing the three areas of interest: sustainability, project development, and financing.
  • Incoming documents: the retrieved PDFs, which are evaluated against the Benchmark using the domain knowledge captured in the Room.

The Room provides the domain knowledge, while the Benchmark defines the criteria used to assess each document’s relevance.

Relevance scoring and project prioritization

Before scoring, documents are preprocessed: stop words are removed, and n-grams preserve meaningful multi-word expressions.

The system uses GloVe word embeddings and cosine similarity to compare document terms with the weighted Benchmark phrases, counting only strong matches. It calculates separate relevance scores for sustainability, project development, and financing, then multiplies the three scores to obtain an overall relevance score. The resulting scores are used to rank potential projects and help financial experts identify documents for further review.

Green-Connect was developed in 2022–2023, before large language models came into widespread use. Today, they could extend the system, for example by identifying Benchmark keywords automatically instead of relying on expert-defined lists.

Results

0.87rank correlation between the system's relevance scores and expert assessments
~6 hoursto search the web and shortlist candidate documents

Evaluation against expert assessments

To evaluate the ranking performance, 70 documents were randomly selected from the shortlisted PDFs and scored by the system. Subject-matter experts independently assessed the relevance of the same documents on a scale of 0 to 10. The system’s scores showed a strong and statistically significant correlation with expert ratings (Spearman’s ρ = 0.87, p < 10⁻²²), indicating that the automated rankings closely followed expert assessments.

Methods & tools

Natural language processing · text mining · word embeddings · semantic similarity · web crawling · information retrieval · document ranking

This research was a collaboration with, and funded by, Siemens Financial Services. My role focused on validating the system against subject-matter experts' assessments.