Converting spoken field observations into editable construction documentation
Company
Lotus Demolition Oy
Industry
- Construction
- Real Estate & Property Management
Problem
Construction site diaries are important operational records and may also be required for legal, contractual and compliance purposes. Site supervisors and foremen need to document daily activities, personnel, subcontractors, weather conditions, work phases, inspections, events, deviations and other relevant observations.
Completing site diaries manually can be slow and inconsistent, particularly when information is entered at the end of the working day from memory. Site personnel may need to recall events, type notes, calls, transfer information into the correct diary fields and prepare the final report after completing their other duties. This creates a risk of missing details, delayed reporting and variation in how information is recorded. It can also make it more difficult to follow project progress and maintain reliable records for compliance, audits, project handovers and communication between site and office personnel.
Our main challenge was therefore to find a practical way for site personnel to record observations by voice during or after the working day, including while travelling from one construction site to another, and convert them into a structured site diary. We wanted to reduce typing and repetitive form completion while keeping the responsible employee in control of reviewing and approving the final record.
Solution
We piloted a Construction Site Diary Assistant that supports knowledge capture by converting spoken field observations into editable construction documentation. The user enters a project ID and uploads an audio file or records a voice note directly in the application.
In the main Diary mode, the AI identifies and extracts specific information from the observation into predefined fields, including personnel, daily work, work phases, events, and other relevant observations. Information not mentioned in the recording is intended to remain blank. The complementary free observation mode converts the recording into a single editable observation without predefined fields.
The user reviews and edits the generated content before creating the final report. Images can be attached, and the completed report can be downloaded as a PDF or sent by email. The application interface is available in Finnish and English.
Evaluation was integrated into the solution development. Haaga-Helia University of Applied Sciences and the University of Helsinki developed suitable evaluation methods, metrics and supporting tools and code for assessing the solution’s transcription and information-extraction output. The methods were used to evaluate and compare results across 12 sample recordings. In addition to validating the company-specific pilot, the work contributed reusable evaluation methods, metrics and tools for the systematic assessment of similar speech-recognition and structured-extraction solutions.
The assistant was developed as part of the GAIK project (https://gaik.ai), which develops a modular open-source GenAI toolkit for organisational knowledge management. The AI workflow reuses components for speech-to-text conversion, optional transcript enhancement and language-model-based extraction of structured information. The solution was created by combining and configuring these reusable components rather than starting from scratch.
Outcome
The pilot demonstrated that the Construction Site Diary Assistant can perform its intended core task of converting spoken site observations into structured, editable diary content. It also supports the capture of individual free-form observations and allows users to review and correct the generated content before producing the final report.
We found that the tool can support the required documentation work and provide a practical foundation for further development. Testing improved our understanding of the technology’s capabilities and helped identify what would be required to develop the pilot into a production system.
Additional testing is required with more varied real-world data to evaluate performance across different speakers, construction terminology, background-noise conditions, recording devices, site environments and types of diary content.
ROI
Quantitative ROI has not yet been established at the pilot stage.
The expected return is based on reducing the time spent typing observations and transferring information into site diary fields. The solution may also reduce repetitive form completion, make report preparation faster and improve the consistency and completeness of daily documentation.
Additional value may come from capturing observations closer to the time when the work occurs, reducing omissions and making structured site information available more quickly for project monitoring, reporting, compliance, audits, archiving and handovers. These benefits remain expected rather than measured.
Financial ROI can be evaluated after the solution has been tested in broader operational use and compared with the existing documentation process. Relevant indicators could include the average time required to complete a site diary, administrative time saved per project, and user satisfaction and continued use.
Technologies
- AI: AI for Speech Recognition & Synthesis, Natural Language Processing (NLP)
- Closed selection: Azure
- Free-text field: GAIK transcriber with Finnish transcript enhancement, GAIK schema generator and extractor, OpenShift Kubernetes (CSC Rahti); FFmpeg; various AI models and speech-recognition approaches, including the Finnish fine-tuned WhisperX (Finnish-NLP/whisper-large-finnish-v3-ct2) model and word-level alignment, and OpenAI models for structured extraction.
Partners
The solution was co-developed with Haaga-Helia University of Applied Sciences as the primary development partner. The University of Helsinki supported testing and evaluation of the transcription output, including ground-truth preparation, while Tampere University conducted an AI adoption readiness assessment. The collaboration provided a structured foundation for development through model comparison, systematic testing and evaluation.
Process Impact
- Facilities & Asset Management
- Operations Management
Languages
- Finnish
- English