# Kotlin & Gmail API - listing emails

## The idea

Let's demonstrate the basic functionality on a useful idea. I have a folder/label in Gmail with a lot of emails. I want to list all email addresses - all senders.

Everything I need to do is a small console app to go through all emails with the given label and extracting the `From` header. If it contains the email address in format `Name <email@address.com>`, extract only the email address.

Let's dive into it!

## Get credentials

Before we start, you need to create a new project in [Google Cloud Console](https://console.cloud.google.com). 

Under the new project, navigate to **API & Services** and enable **Gmail API** in **Library**.

In **Credentials**, click the **+ CREATE CREDENTIALS**, select **OAuth client ID** and setup it like this:

![OAuth client ID setup](https://content.localazy.com/kotlin_gmail_api/console.png)

Click **Save** and download the credentials for your newly created ID:

![OAuth client ID setup](https://content.localazy.com/kotlin_gmail_api/credentials.png)

Save the downloaded file as `credentials.json`.

## Kotlin project

Create a new Kotlin project with Gradle and add Google's dependencies:

```groovy
dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib"
    implementation 'com.google.api-client:google-api-client:1.23.0'
    implementation 'com.google.oauth-client:google-oauth-client-jetty:1.23.0'
    implementation 'com.google.apis:google-api-services-gmail:v1-rev83-1.23.0'
    implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.4.0-M1'
}
```

## Authorization scopes

As we only want to go through labels, messages (we need just headers - metadata), we need a small and safe set of scopes: 

```kotlin
private val SCOPES = setOf(
        GmailScopes.GMAIL_LABELS,
        GmailScopes.GMAIL_READONLY,
        GmailScopes.GMAIL_METADATA
)
```

Beware that if you provide the `GmailScopes.GMAIL_METADATA`, you are not able to access the whole message. You have to omit it if you want to get the message body. 


## Authorize with Gmail

Fortunately, Google libraries come with everything we may need including the server for receiving the authorization request. The whole implementation is as simple as:

```kotlin
private fun getCredentials(httpTransport: NetHttpTransport): Credential? {
    val inputStream = File("credentials.json").inputStream()
    val clientSecrets = GoogleClientSecrets.load(JSON_FACTORY, InputStreamReader(inputStream))
    val flow = GoogleAuthorizationCodeFlow.Builder(httpTransport, JSON_FACTORY, clientSecrets, SCOPES)
            .setDataStoreFactory(FileDataStoreFactory(File(TOKENS_DIRECTORY_PATH)))
            .setAccessType("offline")
            .build()
    val receiver = LocalServerReceiver.Builder().setPort(8888).build()
    return AuthorizationCodeInstalledApp(flow, receiver).authorize("user")
}
```

This code above outputs the request for authorization to the console:

```shell
Please open the following address in your browser:
  https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=...
```

Click the link, authorize the app, and the authorization token is received and stored in `TOKENS_DIRECTORY_PATH`. You only need to do this for the first time. Next time, the stored token is used. 

## Build client  & get labels

We can now use the `getCredentials()` function above to build an authorized client, list all labels, and find the required one identified by `labelName`. 

```kotlin
// Build a new authorized API client service.
val httpTransport = GoogleNetHttpTransport.newTrustedTransport()
val service = Gmail.Builder(httpTransport, JSON_FACTORY, getCredentials(httpTransport))
                .setApplicationName(APPLICATION_NAME)
                .build()

// Find the requested label
val user = "me"
val labelList = service.users().labels().list(user).execute()
val label = labelList.labels
        .find { it.name == labelName } ?: error("Label `$labelName` is unknown.")

```

## List all emails

For listing all email messages, let's use a few of Kotlin's goodies - `tailrec` extension function with lambda as the last parameter. 

We need to invoke the list request repeatedly until the `nextPageToken` is `null`, and doing so with `tailrec` is safer. 

For each message, we invoke the `process` lambda to perform an actual operation.

```kotlin
private tailrec fun Gmail.processMessages(
    user: String,
    label: Label,
    nextPageToken: String? = null,
    process: (Message) -> Unit
) {

    val messages = users().messages().list(user).apply {
        labelIds = listOf(label.id)
        pageToken = nextPageToken
        includeSpamTrash = true
    }.execute()

    messages.messages.forEach { message ->
        process(message)
    }

    if (messages.nextPageToken != null) {
        processMessages(user, label, messages.nextPageToken, process)
    }
        
}
```

## Process message

The code for listing emails above returns only `id` and `threadId` for each of the messages, so we need to fetch message details, extract `From` header, and eventually process it. 

To speed up the process, let's use Kotlin's coroutines to perform the message fetching in parallel. First, introduce a custom dispatcher, so we can limit the number of threads. 

```kotlin
private val MAX_FETCH_THREADS = Runtime.getRuntime().availableProcessors()

val executors = Executors.newFixedThreadPool(MAX_FETCH_THREADS)

val dispatcher = object : CoroutineDispatcher() {
    override fun dispatch(context: CoroutineContext, block: Runnable) {
        executors.execute(block)
    }
}
```

For extracting the email address from the `Name <email@address.com>` format, the code is simple:

```kotlin
private fun String.parseAddress(): String {
    return if (contains("<")) {
        substringAfter("<").substringBefore(">")
    } else {
        this
    }
}
```

Now, we can put things together. Of course, you should introduce some logic for catching exceptions, etc. 

```kotlin
private fun Gmail.processFroms(
        user: String,
        label: Label,
        process: (String) -> Unit
) {
    runBlocking(dispatcher) {
        processMessages(user, label) { m ->
            launch {
                val message = users().messages().get(user, m.id).apply { format = "METADATA" }.execute()
                message.payload.headers.find { it.name == "From" }?.let { from ->
                    process(from.value.parseAddress())
                }
            }
        }
    }
}
```

## Result

With all the code above, we can unique list of all senders like this:

```kotlin
val senders = mutableSetOf<String>()
service.processFroms(user, label) {
    senders += it
}

senders.forEach(::println)
```

## Source code

The complete source code is [available on Github](https://github.com/vaclavhodek/gmail-email-extractor).

---

Originally published on [Localazy](https://localazy.com) - the best developer-friendly solution for localization of web, desktop and mobile apps.