Content Classification Tutorial

Audience

This tutorial is designed to let you quickly start exploring and developing applications with the Cloud Natural Language API. It is designed for people familiar with basic programming, though even without much programming knowledge, you should be able to follow along. Having walked through this tutorial, you should be able to use the Reference documentation to create your own basic applications.

This tutorial steps through a Natural Language application using Python code. The purpose here is not to explain the Python client libraries, but to explain how to make calls to the Natural Language API. Applications in Java and Node.js are essentially similar. Consult the Natural Language API Samples for samples in other languages (including the sample in this tutorial).

Prerequisites

This tutorial has several prerequisites:

• You've set up a Cloud Natural Language project in the Google Cloud console.
• You've set up your environment using Application Default Credentials in the Google Cloud console.
• You are familiar with Python in the Google Cloud console programming.
• You have set up your Python development environment. It is recommended that you have the latest version of Python, pip, and virtualenv installed on your system. For instructions, see the Python Development Environment Setup Guide for Google Cloud Platform.
• You've installed the Google Cloud Client Library for Python

Overview

This tutorial walks you through a basic Natural Language application, using classifyText requests, which classifies content into categories along with a confidence score, such as:

category: "/Internet & Telecom/Mobile & Wireless/Mobile Apps & Add-Ons"
confidence: 0.6499999761581421

To see the list of all available category labels, see Categories.

In this tutorial, you will create an application to perform the following tasks:

• Classify multiple text files and write the result to an index file.
• Process input query text to find similar text files.
• Process input query category labels to find similar text files.

The tutorial uses content from Wikipedia. You could create a similar application to process news articles, online comments, and so on.

Source Files

You can find the tutorial source code in the Python Client Library Samples on GitHub.

This tutorial uses sample source text from Wikipedia. You can find the sample text files in the resources/texts folder of the GitHub project.

Importing libraries

To use the Cloud Natural Language API, you must to import the language module from the google-cloud-language library. The language.types module contains classes that are required for creating requests. The language.enums module is used to specify the type of the input text. This tutorial classifies plain text content (language.enums.Document.Type.PLAIN_TEXT).

To calculate the similarity between text based on their resulting content classification, this tutorial uses numpy for vector calculations.

import argparse
import json
import os

from google.cloud import language_v1
import numpy

Step 1. Classify content

You can use the Python client library to make a request to the Natural Language API to classify content. The Python client library encapsulates the details for requests to and responses from the Natural Language API.