Creating an Apache Hive Table with MongoDB: Creating a MongoDB Data Store

In this section we shall create a MongoDB datastore. We shall store the following sample log (taken from WebLogic Server) in MongoDB.

Apr-8-2014-7:06:16-PM-PDT Notice WebLogicServer AdminServer BEA-000365 Server state changed to STANDBY

Apr-8-2014-7:06:17-PM-PDT Notice WebLogicServer AdminServer BEA-000365 Server state changed to STARTING

Apr-8-2014-7:06:18-PM-PDT Notice WebLogicServer AdminServer BEA-000360 Server started in RUNNING mode

1. Start MongoDB server to be able to access MongoDB using the following command.

mongod

MongoDB gets started. The output from the mongod command is listed:

[root@localhost mongodb]# mongod

mongod –help for help and startup options

2014-06-22T12:l8:56.908-0400

2014-06-22T12:18:56.918-0400 warning: 32-bit servers don’t have journaling enabled by default. Please use –journal if you want durability.

2014-06-22T12:18:56.920-0400

2014-06-22T12:18:57.112-0400 [initandlisten] MongoDB starting : pid=2610 port=27017 dbpath=/data/db 32-bit host=localhost.oraclelinux

2014-06-22T12:18:57.117-0400 [initandlisten]

2014-06-22T12:18:57.119-0400 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.

2014-06-22T12:18:57.124-0400 [initandlisten] ** 32 bit builds are limited

to less than 2GB of data (or less with –journal).

2014-06-22T12:18:57.129-0400 [initandlisten] ** Note that journaling

defaults to off for 32 bit and is currently off.

2014-06-22T12:18:57.130-0400 [initandlisten] ** See http://dochub.mongodb.

org/core/32bit

2014-06-22T12:18:57.136-0400 [initandlisten]

2014-06-22T12:18:57.153-0400 [initandlisten] db version v2.6.3

2014-06-22T12:18:57.167-0400 [initandlisten] git version: 255f67a66f9603c59380b2a389e386910bbb52cb

2014-06-22T12:18:57.203-0400 [initandlisten] build info: Linux ip-10-225-17­11 2.6.18-194.32.1.el5xen #1 SMP Mon Dec 20 11:08:09 EST 2010 i686 BOOST_LIB_ VERSION=1_49

2014-06-22T12:18:57.206-0400 [initandlisten] allocator: system

2014-06-22T12:18:57.206-0400 [initandlisten] options: {}

2014-06-22T12:18:58.582-0400 [initandlisten] waiting for connections on port 27017

2. Next, we shall create a MongoDB data store using a Java application in Eclipse IDE. Start Eclipse IDE from the Eclipse installation directory.

./eclipse

3. Click on File ➤ New. In the New window, select Java>Java Project and click on Next as shown in Figure 11-1.

4. In New Java Project specify a Project name (MongoDB), select Use default location, select a JRE, and click on Next as shown in Figure 11-2.

5. In Java Settings select Allow output folders for source folders. The Default folder name is prespecified as MongoDB/bin as shown in Figure 11-3

6. The Source folder is also preselected as MongoDB/src as shown in Figure 11-4. Click on Finish.

A Java project MongoDB gets created as shown in Figure 11-5.

7. We need to add the MongoDB Java driver Jar file to the project Java Build Path. Right-click on MongoDB project node and select Properties as shown in Figure 11-6.

8. In Properties for MongoDB select Java Build Path and click on Add External JARs to add the MongoDB Java driver jar file mongo-java-driver-2.11.3.jar as shown in Figure 11-7. Click on OK.

9. Next, add a Java class to the Java project. Select File ➤ New and in the New window select the Java ➤ Class wizard and click on Next as shown in Figure 11-8.

10. In New Java Class the Source folder is prespecified as MongoDB/src. Specify a Package name, mongodb. Specify a class Name, CreateMongoDBDocument as shown in Figure 11-9. Select the main method stub to add to the class and click on Finish.

A Java source file CreateMongoDBDocument.java gets added to the MongoDB project as shown in Figure 11-10.

As a summary, to add a document to MongoDB server a connection to MongoDB is first established. Subsequently, a Java class representation of a MongoDB database instance is created, and a Java class representation of a MongoDB collection is created. A MongoDB document Java class representation is created, and the document is added to the collection.

11. A connection to MongoDB is represented with the com.mongodb.MongoClient class. Follow these steps:
    • Create an instance of MongoClient using the MongoClient(List<ServerAdd ress> seeds) constructor.
    • Create a List<ServerAddress> using the host name as 10.0.2.15 and port as 27017.
    • A logical database on the MongoDB server is represented with the com. mongodb.DB class. Create a DB instance using the getDB(String dbname) method in MongoClient with database name as test.
    • The DBCollection class represents a collection of document objects in a database. Create a DBCollection instance using the MongoClient method createCollection(String name, DBObject options).
    • A key-value map for a BSON document object in a database collection is represented with the DBObject interface. The BasicDBObject class implements the DBObject interface and provides constructors to create a document object for a key/value. Create a document object using the BasicDBObject(String key, Object value) constructor and add key/value pairs to the object using the append(String key, Object val) method.
    • An instance of BasicDBObject represents a document object in a database collection. Create three BasicDBObject instances from the log data to be added to the MongoDB database collection.
    • The DBCollection class provides overloaded insert methods for adding a BasicDBObject instance to a collection. Add BasicDBObject instances to a DBCollection using the insert(DBObject… arr) method.
    • To find a document object in the collection, invoke the findOne() method on the DBCollection object.

The CreateMongoDBDocument class is listed:

package mongodb;

import com.mongodb.MongoClient;

import com.mongodb.DB;

import com.mongodb.DBCollection;

import com.mongodb.BasicDBObject;

import com.mongodb.DBObject;

import com.mongodb.ServerAddress;

import java.util.Arrays;

import java.net.UnknownHostException;

public class CreateMongoDBDocument {

public static void main(String[] args) {

try {

MongoClient mongoClient = new MongoClient(

Arrays.asList(new ServerAddress(“10.0.2.15”, 27017)));

DB db = mongoClient.getDB(“test”);

DBCollection coll = db.createCollection(“wlslog”, null);

BasicDBObject row1 = new BasicDBObject(“TIME_STAMP”,

“Apr-8-2014-7:06:16-PM-PDT”).append(“CATEGORY”, “Notice”)

.append(“TYPE”, “WebLogicServer”)

.append(“SERVERNAME”, “AdminServer”)

.append(“CODE”, “BEA-000365”)

.append(“MSG”, “Server state changed to STANDBY”);

coll.insert(row1);

BasicDBObject row2 = new BasicDBObject(“TIME_STAMP”,

“Apr-8-2014-7:06:17-PM-PDT”).append(“CATEGORY”, “Notice”)

.append(“TYPE”, “WebLogicServer”)

.append(“SERVERNAME”, “AdminServer”)

.append(“CODE”, “BEA-000365”)

.append(“MSG”, “Server state changed to STARTING”);

coll.insert(row2);

BasicDBObject row3 = new BasicDBObject(“TIME_STAMP”,

“Apr-8-2014-7:06:18-PM-PDT”)

.append(“CATEGORY”, “Notice”)

.append(“TYPE”, “WebLogicServer”)

.append(“SERVERNAME”, “AdminServer”)

.append(“CODE”, “BEA-000360”)

.append(“MSG”, “Server started in RUNNING mode”);

coll.insert(row3);

DBObject catalog = coll.findOne();

System.out.println(row1);

} catch (UnknownHostException e) {

e.printStackTrace();

}

}

}

12. To add the BSON document objects to MongoDB run the Java application. Right-click on CreateMongoDBDocument.java source file and select Run As ➤ J ava Application as shown in Figure 11-11.

A MongoDB document store is created. One of the document objects added to the document store gets output in the Eclipse IDE Console as shown in Figure 11-12.

Each BSON document object has an _id key added automatically.

{ “_id” : { “$oid” : “53a6cf25e4b09cac451ef1d6”} , “TIME_STAMP” : “Apr-8- 2014-7:06:16-PM-PDT” , “CATEGORY” : “Notice” , “TYPE” : “WebLogicServer” , “SERVERNAME” : “AdminServer” , “CODE” : “BEA-000365” , “MSG” : “Server state changed to STANDBY”}

Source: Vohra Deepak (2015), Pro MongoDB™ Development, Apress; 1st ed. edition.