Use Case – Safety Observation Reporting Assistant

Recorded or typed safety observations converted into a report that can be reviewed, corrected and exported

a man reporting a hazard to a mobile at a consturction site

Company

Lotus Demolition Oy

Industry

  • Construction
  • Real Estate & Property Management

Problem

At construction and demolition sites, employees are expected to identify and report safety observations, including situations where a potential risk is identified before an incident occurs. According to company policy, each employee should submit at least one safety observation per month. Across the company, this can result in more than 1,000 observations per year. 

Everyone in the company can act as an observer and submit observations. The reports are then used by the HSEQ Manager and other managers to monitor safety issues and support follow-up actions. The high reporting volume creates a need for an efficient and consistent way to capture observations and convert them into complete, structured reports. 

Solution

The Safety Observation Reporting Assistant allows employees to record or type a safety observation and converts the input into a report that can be reviewed, corrected and exported. It supports two reporting modes: a structured safety observation report created from audio or text, and a free-form observation generation from audio. Users can also attach up to five supporting images.  

In the structured mode, speech is transcribed and AI extracts the information into the required safety observation fields. In the free-form mode, audio is transcribed into a single observation field, or typed text is stored unchanged without LLM-based structuring. Before the report is used, a person reviews and edits the information. Reports can be downloaded as PDF, JSON, CSV or text. In the pilot, PDFs can also be saved to CSC Allas or sent by email. The pilot uses enterprise cloud services from Microsoft Azure and Google Cloud, where customer data is not used to train AI models without permission; privacy and security settings must be confirmed before production use 

The assistant was developed as part of the GAIK project (https://gaik.ai), which develops a modular open-source GenAI toolkit for organisational knowledge management. During use, the GAIK Transcriber converts the voice recording into text, after which the Data Extractor identifies the required information and places it into the appropriate safety observation fields. The company-specific solution was designed in one workshop by reusing GAIK’s generic incident-reporting solution and its customisation process. At design time, the Requirement Parser and Schema Generator were used to convert Lotus Demolition Oy’s reporting requirements into a reusable data structure. Lotus Demolition Oy provided the required forms, reporting guidelines, sample data and testing feedback. The solution was co-developed with Haaga-Helia University of Applied Sciences.

Outcome

Our initial experience with the tool was positive. We conducted several tests and found that the tool could perform the required task. The testing also helped us better understand the solution’s capabilities and identify possible directions for further development. 

Because testing has so far been limited, we cannot yet draw broader conclusions about the solution’s performance. Additional testing and evaluation with more varied data are currently in progress to assess how consistently the solution works in different situations. Overall, the pilot has demonstrated promising initial results, and we are planning the next steps to move it closer to production. 

ROI

Quantitative ROI has not been measured. The expected value comes from reducing manual form-filling, supporting more complete and consistent reports, and making safety observations available for follow-up sooner. Possible future indicators are report completion time, completeness of required fields and the number of observations processed. 

Technologies

  • AI: AI for Speech Recognition & Synthesis, Natural Language Processing (NLP) 
  • GAIK transcriber with Finnish speech transcript enhancement, GAIK schema generator and structured extractor, Azure OpenAI; OpenShift Kubernetes (CSC Rahti); CSC-hosted S3-storage (Allas); FFmpeg; various AI models and speech-recognition approaches, including Finnish fine-tuned WhisperX (Finnish-NLP/whisper-large-finnish-v3-ct2) model and word-level alignment, OpenAI Whisper Large, OpenAI GPT-4o Transcribe, and Google Gemini 3

Partners

The solution was co-developed with Haaga-Helia University of Applied Sciences as the primary development partner.  

Process Impact

  • Facilities & Asset Management
  • Risk Management & Compliance

Languages

  • Finnish
Scroll to Top