How to Download Android Texts for Free

I’ve been working on a website to build word clouds (wordcloudgifts.com) and wanted to make a word cloud from texts I found on a couple of old phones. I quickly found out that this process is not as straightforward as moving pictures off an old device. Here’s how you can do it by building a small application yourself.

I tested on my Google Pixel (version 9), Moto X (version 5.1), and Samsung Galaxy Nexus (version 4.2)

How Text Messages are Stored

Text messages and contact data are stored in database files on your Android phone. For older phones the location is:

/data/data/com.android.providers/telephony/databases/mmssms.db

For any Android phone after version KitKat, you can find them in this location:

/data/data/com.android.providers.telephony/databases/mmssms.db

If you don’t see the file at either location you may have better luck searching with this command:

find / -name "*mmssms*"

One option to download your text messages is to directly find and transfer these files to your computer. You can then open and query with sqlite. However, you won’t be able to find these files by simply looking at hidden files in file explorer. You’ll need another method to explore as root. Instead, I opted to create a very simple application to export this data as a csv.

Android Studio

For this method you will be using Android Studio. You can download it here. Android Studio is a behemouth of an install and setup so it’ll take a while. Download the SDK versions that you need for your phones.

Connect to Your Phone

You will need to set up your Android phone to enable debug mode in order to develop with Android Studio. For most phones, simply go to settings, click on the version number 7 times to show developer settings, then under developer settings you can enable debug mode. If this doesn’t work for your phone, there are plenty of tutorials on how to enable debug mode for each Android version.

Create a Simple Android Application

You can find the full application code here: https://github.com/Will-Renius/download-sms

Once you’re set up to develop for Android, you can create a simple appliction. You will get a list of new templates to use.

Click blank template. If you want to build out a UI, you can view the full application code.

Run the program and you should see a hello world screen pop up on your phone.

Add OpenCsv

I’ll be exporting as a csv. It’s pretty straightforward to create a csv if you don’t have to deal with escaping. However, if you ever included a “,” new line or quotes in a text message, it’ll be easier to use a package built for csvs.

Download from here

You’ll need to paste the jar into your project in order to use the OpenCsv package. I followed this tutorial on Stack Overflow: https://stackoverflow.com/questions/25660166/how-to-add-a-jar-in-external-libraries-in-android-studio

Building the App

If you’re following along and didn’t already have Android Studio setup, you’re through with the hard part. Setup is complete and now we just have to write a couple scripts to access the right databases.

Permissions

First you’ll make sure your application can read SMS messages. Change the manifest.xml file to include the following permissions:

    <uses-permission android:name="android.permission.READ_SMS" />
    <uses-permission android:name="android.permission.READ_CONTACTS" />
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>

On Android versions earlier than API version 23, this change is sufficient.

For later devices we need to build in some custom logic to prompt users for permissions at runtime. Here is the condition we’ll need to check for all permissions:

// Request codes. Can be any number > 0.
private static final int PERMISSIONS_REQUEST_ALL = 42;

// Check the SDK version and whether the permission is already granted or not.
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M && 
(
        checkSelfPermission(Manifest.permission.READ_SMS) != PackageManager.PERMISSION_GRANTED ||
        checkSelfPermission(Manifest.permission.READ_CONTACTS) != PackageManager.PERMISSION_GRANTED ||
        checkSelfPermission(Manifest.permission.READ_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED ||
        checkSelfPermission(Manifest.permission.WRITE_EXTERNAL_STORAGE) != PackageManager.PERMISSION_GRANTED)
){
    requestPermissions(new String[]{Manifest.permission.READ_SMS,
            Manifest.permission.READ_CONTACTS,
            Manifest.permission.READ_EXTERNAL_STORAGE,
            Manifest.permission.WRITE_EXTERNAL_STORAGE}, PERMISSIONS_REQUEST_ALL);
//          After this point wait for callback in onRequestPermissionsResult(int, String[], int[]) overriden method
} else {
    // run code
}

Imports

Here are the packages we’ll be using:

package com.example.myapplication5; // the application name

import androidx.appcompat.app.AppCompatActivity;

import android.content.Context;
import android.os.Environment;
import android.database.Cursor;
import android.net.Uri;
import android.os.Bundle;

import com.opencsv.CSVWriter;

import java.util.*;
import java.io.*;

Reading Your Messages

The important tables for reading SMS messages are inbox, sent, and contacts. Inbox will only give you messages you’ve received, sent will give you the messages you sent to others. Neither of these tables will give you the names of people, so we need to use contacts to retrieve names from ids.

To read from inbox we use a cursor class. Here I’m only selecting a few columns. If you want to view them all, simply leave the second parameter as null.

Cursor inbox_cursor = getContentResolver().query(Uri.parse("content://sms/inbox"), new String[]{"thread_id","body","person", "date"}, null, null, null);

We want to convert the table data into a list, where we can write it as csv. With the cursor populated, you can loop through each row in the inbox table and add values to our list.

List<String[]> inbox_data = new ArrayList<String[]>();
inbox_data.add(new String[] {"thread_id", "person", "date", "body"});
if (inbox_cursor.moveToFirst()) { // must check the result to prevent exception
    do {
        String thread_id = inbox_cursor.getString(0);
        String body = inbox_cursor.getString(1);
        String person = inbox_cursor.getString(2);
        String date = inbox_cursor.getString(3);

        inbox_data.add(new String[] {thread_id, person, date, body});
    } while (inbox_cursor.moveToNext());
} else {
    // empty box, no SMS
}

// Close the cursor
inbox_cursor.close();

We can combine this logic into a function and get the following:

/**
    * Get columns of sent messages
    *
    * @return a list of separated column values.
    */
private List<String[]> getInboxSMS() {
    List<String[]> inbox_data = new ArrayList<String[]>();

    Cursor inbox_cursor = getContentResolver().query(Uri.parse("content://sms/inbox"), new String[]{"thread_id","body","person", "date"}, null, null, null);
    inbox_data.add(new String[] {"thread_id", "person", "date", "body"});
    if (inbox_cursor.moveToFirst()) { // must check the result to prevent exception
        do {
            String thread_id = inbox_cursor.getString(0);
            String body = inbox_cursor.getString(1);
            String person = inbox_cursor.getString(2);
            String date = inbox_cursor.getString(3);

            inbox_data.add(new String[] {thread_id, person, date, body});
        } while (inbox_cursor.moveToNext());
    } else {
        // empty box, no SMS
    }

    // Close the cursor
    inbox_cursor.close();

    return inbox_data;
}

Once we have our data arrays constructed, we can use openCSV to write csv files to the file system. I’m setting everything up to write to the downloaded folder, so it is easy to navigate to and move to your computer.

String baseDir = android.os.Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS).getAbsolutePath();
String fileName = "export";

//        Write inbox CSV
String inbox_filename = fileName + "_inbox_sms.csv";
String inbox_filePath = baseDir + File.separator + inbox_filename;

try {
    CSVWriter writer = new CSVWriter(new FileWriter(inbox_filePath));
    writer.writeAll(inboxSMS);
    writer.close();
} catch (IOException e) {
    e.printStackTrace();
    Toast.makeText(this, "Download Failed", Toast.LENGTH_SHORT).show();
}

Rinse and repeat for sent messages (content://sms/sent) and contacts (content://contacts/people) to download all needed csv files. You can view the full code together here: https://github.com/Will-Renius/download-sms

Once you run this code on your phone, you can use your file manager to access the csv files in your downloads folder.

Parsing Texts

You now have three datasets. You could try to merge them within your Android app and save a single csv, but I find it much easier to work with another language such as R or Python. Here is how you can manipulate and save the data you want in Python. If you wanted to get all the text messages you shared with your mom, you could run the following code:

import pandas as pd

contacts = pd.read_csv('contacts.csv')
inbox = pd.read_csv('inbox_sms.csv')
sent = pd.read_csv('sent_sms.csv')
# Get all the threads with only specific person on. Then get a list of all body messages sent and received within that thread
named_inbox = inbox.merge(contacts, left_on='person', right_on='id')
named_inbox[named_inbox["display_name"] == "Mom"]["thread_id"].unique()
pd.concat([sent[sent["thread_id"] == 4], named_inbox[named_inbox["thread_id"] == 4]], sort=False)["body"].to_csv("mom_texts.csv", header=False, index=False)

Using wordcloudgifts.com, I was finally able to make the wordcloud of texts I set out to make.