Towards the Primary Platform for
Language Technologies in Europe

Choose the right tool to create your ELG service in Python

With the Python SDK, the European Language Grid provides tools to facilitate the creation of an ELG service from your Language Technology (LT) tool in Python. It is very easy to convert your LT tool running on your computer to an ELG service accessible and usable by everyone. In this blog post, we will show you the different options that the Python SDK offers for building your ELG service, and when to use them.

The ELG Python SDK

Before we start, a little reminder on what the ELG Python SDK is and what does.

The Python SDK is a pip package and can be installed easily via pip/PyPI:

pip install elg

It provides access to most of the ELG functionalities in Python. It provides access to the catalogue of resources with methods that allow you to search the catalogue and look for corpora, services, and information on organisations. The Python SDK allows you to call the services available into ELG and even combine them with a pipeline mechanism. In addition, and this is what interests us in this blog post, the Python SDK also contains utilities to build ELG compatible services.

If you want to learn more about the Python SDK, you can have a look at the documentation.

ELG compatible services

To create an ELG service, it is not necessary to understand what a service in the ELG is, but it does help, so we will explain it briefly here. Note that if you want more details on the ELG architecture, it is detailed in several publications, as well as in our documentation.

ELG services are Docker containers that run in ELG’s Kubernetes cluster. Not all services are running all the time, i.e. not all Docker images of services are launched; in fact, we use knative to run them on demand, only when users want to use them. These Docker containers (which correspond to the ELG services) are not accessible from outside the cluster and communicate only with the REST server (LT Service Execution Orchestrator). The communication between the ELG services and the REST server is done according to the specification of the ELG Internal LT Service API – we will come back to this point a little later in this post. When we use an ELG service, the request is therefore made to the REST server which then forwards the request to the service, and returns the response obtained; only the REST server is accessible from outside the cluster.

Technical architecture of the ELG

Technical architecture of the ELG

To recap: an ELG service is a Docker container which communicates with the REST server via the ELG Internal LT Service API. Starting from this, there are three options for integrating your LT tool:

Option 1: Your LT tool is in the Docker image

This is the most common way of creating an ELG service based on your LT tool: the docker image that exposes an ELG compatible endpoint contains your LT tool and runs it.

Option 2: Your LT tool is already a Docker image

If your LT tool is already packed in a Docker image that exposes an HTTP endpoint but is not compatible with ELG, the ELG service can simply be an adapter that will be compatible with the ELG Internal LT Service API, and that will forward the requests to your LT tool Docker container also running inside the ELG cluster (technically, the LT tool and the ELG service Docker containers run in the same Kubernetes pod).

Option 3: Your LT tool is running outside of ELG

Similar to the second option, if your LT tool is already exposing an HTTP endpoint outside of ELG, the ELG service deployed into the ELG cluster can simply be a proxy that will be compatible with the ELG Internal LT Service API, and that will forward the requests to your LT tool running outside of ELG. This option is used when you don't want your LT tool to be deployed into ELG.

The three different options for integrating a LT tool into ELG

The three different options for integrating a LT tool into ELG

Creation of your ELG service using the Python SDK

To create an ELG service, you must create a Docker image that exposes an HTTP endpoint compatible with the ELG Internal LT Service API (this Docker image may or may not contain your LT tool depending on the integration option you have chosen). This can be done relatively easily for most programming languages, using Flask or FastAPI for Python, or Spring for Java for example. However, this always takes a little time and can be complex to optimize depending on your LT tool and the integration option chosen. Fortunately, the ELG Python SDK simplifies this step of creating the Docker image compatible with ELG.

The Python SDK contains two classes: the FlaskService and the QuartService classes, respectively based on Flask and Quart that allow you to easily create an ELG service. To understand how it works, let's look at an example.

These classes are not installed by default, you need to install the ELG pip package with extra dependencies to use them:

pip install elg[flask] # to use the FlaskService class
pip install elg[quart] # to use the QuartService class

Create an ELG service using the FlaskService class

Imagine your LT tool is a language detection tool that runs as follows:

import langdetect

results = langdetect.detect_langs("This is a sentence in English.")
print(results)
>> [en:0.999998356414909]

To convert your tool in an ELG service using the FlaskService class, it is as easy as creating this Python script:

from elg import FlaskService
from elg.model import TextRequest, AnnotationsResponse
import langdetect

class ELGService(FlaskService):
    def process_text(self, content: TextRequest):
        langs = langdetect.detect_langs(content.content)
        ld = {}
        for l in langs:
            ld[l.lang] = l.prob
        return AnnotationsResponse(features=ld)

service = ELGService("LangDetection")
app = service.app

We comment each part of this file so that you can understand how it is working:

from elg import FlaskService

We import the FlaskService class from the ELG Python SDK. This class implements a Flask web server so you only have to redefine the process_text, process_audio, or process_structured_text method depending on the input type of your service.

from elg.model import TextRequest, AnnotationsResponse

Here, we import two classes that represent the request and the response messages of our ELG service. These classes correspond to the specification of the ELG Internal LT Service API, so by using them you ensure that your ELG service will be ELG compatible. Here, the LT tool takes text as input and returns an annotation response, but all the ELG message types are available.

import langdetect

This import is needed to run the LT tool. It is used to perform the language identification.

class ELGService(FlaskService):

We create our ELG service class that inherits from the FlaskService class to take advantage of what is already implemented in the FlaskService class.

    def process_text(self, content: TextRequest):

We redefine the process_text method because the ELG service takes text in input. The content parameter of this request contains all the data of the input request.

        langs = langdetect.detect_langs(content.content)
        ld = {}
        for l in langs:
            ld[l.lang] = l.prob

This part corresponds to the LT tool. We are performing the language identification of the input content. We are changing the form of the output to facilitate the creation of the output message.

        return AnnotationsResponse(features=ld)

Now we return the output of the LT tool as an AnnotationsResponse message to make sure that the ELG service is ELG compatible.

service = ELGService("LangDetection")
app = service.app

Finally, we instantiate the class and define the app variable that will be the endpoint of the Docker container.


After having created the elg_service.py file, you can run the following command:

elg docker create -n ELGService -p elg_service.py -r langdetect

This will create all the needed files to build the Docker image of your ELG service. Among those files, there is the Dockerfile from which you can build the image running the usual docker build …

A more precise example of how to use the FlaskService class is available in our documentation.

The image that we just built is an ELG service that can be deployed in the ELG cluster. Easy, right? We hope so, and if not, please feel free to send us your feedback. We are happy to hear your thoughts.

Choose the right tool depending on your use case

Depending on the LT tool you want to integrate into ELG and the integration option you choose, the requirements for the Docker image of the ELG service might differ. The FlaskService class we just presented is suitable when the LT tool is inside the Docker image (integration option 1), but is not when the LT tool is running outside of the Docker image (options 2 & 3) because the Flask server is synchronised, which means it will have to wait for the response from the LT tool before being able to handle another request. To solve this issue, we created the QuartService class which works very similar to the FlaskService class, except asynchronously. Therefore, the QuartService class is recommended when the ELG service is only an adapter or a proxy (integration options 2 & 3). In addition, the QuartService class has the possibility to stream the request without storing it locally, which can be very useful when dealing with large input data like audio for example*.

The strength of the FlaskService and the QuartService classes is that they work almost the same way, and so you can optimize your ELG service very easily without having to do a lot of configurations. For example, here is the Python script for an ELG Speech-to-Text service that proxies the request to the LT tool running outside of ELG without caching the audio file locally:

import traceback
import aiohttp

from elg import QuartService
from elg.model import TextsResponse
from elg.quart_service import ProcessingError

class Proxy(QuartService):

    consume_generator = False

    async def setup(self):
        self.session = aiohttp.ClientSession()

    async def shutdown(self):
        if self.session is not None:
            await self.session.close()

    async def process_audio(self, content):
        try:
            # Make the remote call
            async with self.session.post("https://example.com/endpoint", data=content.generator) as client_response:
                status_code = client_response.status
                content = await client_response.json()
        except:
            traceback.print_exc()
            raise ProcessingError.InternalError('Error during the call')

        if status_code >= 400:
            raise ProcessingError.InternalError('Error during the call')

        return TextsResponse(texts=[{"content": content["text"]}])

service = Proxy("Proxy")
app = service.app


We try to make the creation of an ELG service as simple as possible with the ELG Python SDK. You can convert your Python LT tool in a Docker image that exposes an ELG compatible endpoint quickly and easily by following the steps described above.

We hope you enjoyed this blog post – if you have questions or feedback on the ELG Python SDK, please feel free to contact us.


* The streaming of the request is currently only supported for audio requests.


Survey for LT developers and users: Shape the future of European Language Technology

Despite the recognizable advantages and historical and cultural worth of multilingualism, the many European languages face a striking imbalance in terms of their preservation in the digital world and their support by language technology. The European Language Equality project (ELE) addresses this risk to European identity in the digital age by preparing a Strategic Research and Innovation Agenda and Roadmap working towards digital language equality by 2030. The European Language Grid (ELG) is closely related to this project, offering LT developers, researchers and providers an inclusive platform to present, share and market their language technologies and connect within the European LT community.

As part of the two projects that are funded by the European Commission and address an appeal by the European Parliament resolution titled “Language equality in the digital age”, we are reaching out both to LT developers and LT users to participate in a large-scale, EU-wide consultation that will impact and shape the future of language technologies in the multilingual continent. The two surveys are aimed on the one hand at academic and commercial developers in the field of Language Technology (LT), Natural Language Processing (NLP) and Language-centric Artificial Intelligence (AI) and on the other at all Language Technology users and consumers.

The questionnaire takes approximately 20 minutes to fill in; your answers will help evaluating the level of LT support for European Languages, indicating the challenges and highlighting the needs and expectations of professionals and users in the future. Your contributions will be carefully taken into account when preparing the ELE strategic agenda and roadmap.

The European Language Equality project is a pan-European effort that will significantly impact the field and funding situation of LT in Europe for the next 10 to 15 years. Help us shape the future of multilingualism in the digital age – join in!


  • Survey for Language Technology developers

  • Survey for Language Technology users and consumers

  • Second Open Call for Pilot Projects closed with 103 submissions

    The second ELG open call for pilot projects was closed on 30 November 2020. In total, 103 project proposals were accepted for evaluation with applicants from 28 countries.

    We received slightly more project proposals from SMEs (60) than research organisations (43). The majority of the proposals focus on contributing resources, services, tools, or data sets to the European Language Grid platform to increase its coverage (66), while there are 37 proposals developing applications using language resources and technologies available in the ELG platform.

    All proposals will be evaluated by three independent experts and by the ELG Pilot Board. We expect to announce the results of the evaluation and selection process by mid-February 2021. The selected projects will start in March 2021.

    In both calls combined, we received 213 project proposals from 156 different institutions (86 SMEs, 70 research organizations) coming from 32 different countries (including 9 eligible countries outside the European Union). Total requested amount is more than 30 mil. EUR, whilst available funding amounts 1.95 mil. EUR.

    As in the first open call, we are happy about this enormous interest in the European Language Grid and the Language Technology topic in general. Good luck to all proposers!

    For more information on the ELG open calls, don’t hesitate to contact us at pilot-projects@european-language-grid.eu.


    Second Open Call – Webinar for Applicants

    The Webinar for Applicants in the Second ELG Open Call for pilot projects will take place online on Thursday, 12 November 2020, 10.00 – 11.30 (CET). At the meeting, we will talk about the goals of the open call and
    briefly present the call documentation and its annexes. The second part of the webinar will be dedicated to questions and discussion. You can share your questions and topics for discussion with us in advance at
    pilot-projects@european-language-grid.eu.

    We will share the link to this meeting with project managers that have created their account in our Open Calls Platform. For information on how to register to the Open Calls Platform and on the Second Open Call, please go to the Open Call #2 website.


    Second Open Call for Pilot Projects published

    The second and last ELG open call for pilot projects has just been published. The submission period will last for two months until the end of November and the proposal submission deadline is 30 November 2020. The total funding sum available for this call is 585,000 EUR. We expect to select 3 to 5 project proposals that will start their execution phase in February 2021.

    For more information see the call documentation on the Open Call #2 page.

    Also, please note that ELG is still looking for external evaluators to evaluate project proposals submitted in the open calls for the pilot projects.


    Second Open Call will Open in October 2020

    After a very successful first open call for pilot projects, open call no. 2 is now in preparation. It will be opened on 1 October 2020 — one month later than originally planned. The submission period will last for two months until the end of November and the proposal submission deadline is 30 November 2020. The total funding sum available for this call is 585,000 EUR. We expect to select 3 to 5 project proposals that will start their execution phase in February 2021.

    The basic parameters for the second open call remain the same: We will look for pilot projects that shall broaden the ELG’s portfolio of language technologies. The projects will develop missing services or solutions that support underrepresented languages. At the same time, the pilot projects should demonstrate the usefulness of the European Language Grid as a technology platform. The amount awarded to each project can be up to 200,000 EUR, and the runtime of the projects is expected to be between 9 and 12 months. SMEs and research organisations are eligible to apply and the projects will be non-consortial, i.e., only one single organisation or institution is allowed per project.


    Results of the Open Call for Pilot Projects #1

    It is a great pleasure for us to announce the results of the first ELG open call for pilot projects.

    The Open Call #1 was closed on 30 April 2020. In total, 110 project proposals were accepted for evaluation, with applications coming from 29 countries. All proposals were evaluated by three independent experts and by the ELG Pilot Board.

    The ELG Pilot Board has selected 10 projects, amounting to a funding sum of EUR 1,363,915 in total. You can find more information on the call as well as the list of selected projects here. The selected projects will now sign the Third Party Agreement and start their execution phase.

    All applicants will receive a brief feedback on their project proposal in the next weeks. If the project proposal was not accepted in the first call, a resubmission in the second call is possible. The second ELG open call for pilot projects will be opened in September or October 2020.

    Thank you to all applicants for the effort dedicated to the first call and for submitting their project proposals. We appreciate the enormous interest in the European Language Grid and hope for future collaboration with all interested parties. Thank you also to all independent evaluators involved for their work.

    Congratulations to all successful applicants – the competition was tough!

    For more information on the ELG open calls, don’t hesitate to contact us at pilot-projects@european-language-grid.eu.


    Enormous interest in the first ELG Open Call for Pilot Projects

    The first ELG open call for pilot projects was closed on 30 April 2020. In total, 110 project proposals were accepted for evaluation with applicants from 29 countries.

    We received slightly more project proposals from SMEs (62) than research organisations (48). The majority of the proposals focus on contributing resources, services, tools, or data sets to the European Language Grid platform to increase its coverage (79), while there are 31 proposals developing applications using language resources and technologies available in the ELG platform.

    All proposals will be evaluated by three independent experts and by the ELG Pilot Board. We expect to announce the results of the evaluation and selection process by the end of June 2020. The selected projects will start in July 2020.

    The whole ELG team is excited about this enormous interest in the European Language Grid and the Language Technology topic in general. Good luck to all proposers!

    For more information on the ELG open calls, don’t hesitate to contact us at pilot-projects@european-language-grid.eu.


    Deadline for projects of First Open Call confirmed

    We are pleased by the great interest in the first ELG open call for pilot projects so far. The deadline for submitting your projects is 30 April 2020.

    As mentioned in the call documentation, we have no specific priorities in this open call. At the same time, projects related to the current COVID-19 situation are more than welcome.

    Please note that we updated the Frequently Asked Questions with new ones that have been raised since the First Open Call was opened.

    If you have any questions, please get in touch with us at
    pilot-projects@european-language-grid.eu.


    First Call for Pilot Projects Published

    Within the ELG project, we will run two open calls. The first call was opened on 1 March 2020, projects can be submitted until 30 April 2020. The second call for pilot projects will be opened in September 2020.

    The projects should last 9–12 months and can be funded with a budget of up to €200,000. They should contribute resources, services, tools or datasets to the ELG to increase its coverage, or they should develop applications using language resources and technologies available in the ELG. SMEs and research organisations are eligible for project funding. Proposals are to be submitted via the ELG Open Calls Platform.

    This is a good opportunity for interested parties to boost their language technologies and to become part of the European Language Grid.

    For more information see the call documentation and the Open Calls page on the ELG website.