Database on Time Friend

MongoDB Migration - Backup and Restore

Wed, 27 Mar 2019 00:00:00 +0000

Recently did a migration of my crawler database, so here record MongoDB backup and restore

MongoDB Data Backup

mongodump -h dbhost -d dbname -o dbdirectory

-h: MongoDB server address, e.g.: 127.0.0.1, if not changed generally is: 127.0.0.1:27017
-d: Database name to backup, e.g.: test
-o: Backup data storage location, e.g.: /home/root/xxx,

MongoDB Data Restore

mongorestore -h <hostname><:port> -d dbname <path>

–host <:port>, -h <:port>: MongoDB server address, default: localhost:27017

MongoDB - Using MongoDB in Egg

Fri, 18 May 2018 00:00:00 +0000

MongoDB official provides node database operation driver and API: node-mongodb-native

In egg community there’s also an egg plugin based on this plugin secondary encapsulation egg-mongo-native

Made some secondary encapsulation of some methods, can also call original methods.

Configuration

Official documentation also has related configuration instructions, but I encountered some problems in actual use. Below I provide correct configuration method for everyone to use. Other related knowledge please refer to this plugin’s official documentation.

MongoDB - Document Basic Operations (Part 2)

Sun, 13 May 2018 00:00:00 +0000

Document Update

Syntax: update(, , )

# Update rank field
> db.posts.update({"title":"Monster Hunter World Review"}, {$set: {"rank": 10} });

# Update entire data to: {"rank": 99}
> db.posts.update({"title":"Monster Hunter World Review"}, {"rank": 99});

# Update multiple records multi: true, if false, then update first record found
> db.posts.update({"tag":"it"}, {$set: {"rank": 60}}, {multi: true});

Document Field Operation Functions

$inc: Increment
$mul: Multiply
$rename: Rename
$set: Add or modify
$unset: Field delete

# Field value increment, result: field value + 1
> db.posts.update({title:"Monster Hunter World Review"}, {$inc: {rank: 1}});

# Field value multiply operation, result: field value * 2
> db.posts.update({title:"Monster Hunter World Review"}, {$mul: {rank: 2}});

# Field rename
> db.posts.update({title:"Monster Hunter World Review"}, {$rename: {"rank": "score"}});

# Set or add field
> db.posts.update({title:"Monster Hunter World Review"}, {$set: {"istop": true}});

# Delete istop field
> db.posts.update({title:"Monster Hunter World Review"}, {$unset: {"istop": true}});

Document Special Updates

upsert: Update if exists, append if not
remove: Conditional delete data

$ mongo
# Update entire data
> db.posts.update({title:"Actually Creation is More Fun than Taishi"}, {title:"Actually Creation is More Fun than Taishi", "rank":5,"tag":"game"});

# Update or add data
> db.posts.update({title:"Actually Creation is More Fun than Taishi"}, {title:"Actually Creation is More Fun than Taishi", "rank":5,"tag":"game"}, {upsert:true});

# Delete data
> db.posts.remove({title:"Actually Creation is More Fun than Taishi"});

Index

getIndexes() Get indexes
createIndex({…}, {…}) Add index
dropIndex({…}) Delete index

# Get indexes
> db.posts.getIndexes();

# Create rank descending index
> db.posts.createIndex({rank:-1});

# Delete index
> db.posts.dropIndex({rank:-1});

# Create an ascending index, and cannot duplicate, subsequent title duplicates will fail to add
> db.posts.createIndex({title:1}, {unique:true});

MongoDB - Document Basic Operations (Part 1)

Sat, 12 May 2018 00:00:00 +0000

Document Add/Delete

# Create collection
> db.createCollection("posts");

# Insert one data
> db.posts.insert(
... {
... title: "My First Blog",
... content: "Started writing blog, so excited."
... }
... );

# Query data
> db.posts.find();

# Insert another data
> db.posts.insert(
... {
... title: "My Second Blog",
... content: "What should I write?",
... tag: ["Uncategorized"]
... }
... );



# Use js for loop to insert data
> for(var i = 3; i <=10; i++ ) {
... db.posts.insert({
... title: "My " + i + "th Blog"
... });
... }

# Query data
> db.posts.find();

# View data count
> db.posts.count();

# Delete all collection data
> db.posts.remove({});

> db.posts.find();

Document Conditional Query

# Query data with tag as game
> db.posts.find({"tag": "game"});

# Query data with rank greater than or equal to 4
> db.posts.find({"rank": {$gte: 4}});

# Query data with rank greater than 4
> db.posts.find({"rank": {$gt: 4}});

# Query data with rank less than or equal to 4
> db.posts.find({"rank": {$lte: 4}});

# Query data with rank less than 4
> db.posts.find({"rank": {$lt: 4}});

# Regular expression query
> db.posts.find({"title": /u/});

# Multi-condition query
> db.posts.find({"title": /u/, "rank":{$gte:5} });

# $or OR condition query
> db.posts.find({$or: [{"title": /u/}, {"rank":{$gte:4}}] });

# $in contains, query data with rank containing 3,4
> db.posts.find({"rank": {$in: [3,4]} });

# Check if exists
> db.posts.find({"istop": {$exists: true} });

# Query data with three tags in article
> db.posts.find({'tag': {$size: 3}});

# Get all tag types, distinct function is to get non-duplicate values of specified field in collection, return as array
> db.posts.distinct("tag");

MongoDB vs RDBMS Where Statement Comparison

Operation	Format	Example	Similar Statement in RDBMS
Equal	{:}	db.col.find({“by”:“test”}).pretty()	where by = ’test'
Less than	{:{$lt:<value%}}	db.col.find({“likes”:{$lt:50}}).pretty()	where likes < 50
Less than or equal	{:{$lte:<value%}}	db.col.find({“likes”:{$lte:50}}).pretty()	where likes <= 50
Greater than	{:{$gt:<value%}}	db.col.find({“likes”:{$gt:50}}).pretty()	where likes > 50
Greater than or equal	{:{$gte:<value%}}	db.col.find({“likes”:{$gte:50}}).pretty()	where likes >= 50
Not equal	{:{$ne:<value%}}	db.col.find({“likes”:{$ne:50}}).pretty()	where likes != 50

Specify Extract Fields

Syntax: db.[collection_name].find({}, {field1: true, field2: 1})

MongoDB - Database and Collection Basic Operations

Fri, 11 May 2018 00:00:00 +0000

Database Basic Operations

# Enter command line
$ mongo

# Help
> help

# Exit
> exit

# Show all databases
> show dbs;

# Enter or create collection
> use aliliblog;

# View current database status
> db.stats();

# Delete database
> db.dropDatabase();

Collection Operations



# View collections
> show collections;

# Create collection
> db.createCollection("users");

# Collection rename
> db.users.renameCollection("staff"); // users -> staff

# Collection delete
> db.staff.drop();

MongoDB - Basic Concepts and Comparison with Other Databases

Thu, 10 May 2018 00:00:00 +0000

Basic Concepts

NoSql

In NoSql databases, data operations are all completed through commands or programming languages.

In MongoDB uses JavaScript and JSON data structures to operate and manage data.

MongoDB Database vs Relational Database Comparison

MongoDB	Relational Database
Database (Database)	Database (Database)
Collection (Collection)	Table (Table)
Document (Document)	Record (Record)

MongoDB vs RDBMS Corresponding Terms

MongoDB	RDBMS
Database	Database
Collection	Table
Document	Row
Field	Column
Embedded Document	Table Join
Primary Key (MongoDB provides key as _id )	Primary Key

MongoDB Data Types

Data Type	Description
String	String. Common data type for storing data. In MongoDB, UTF-8 encoded strings are legal.
Integer	Integer value. Used to store numeric values. Depending on server you adopt, can be 32-bit or 64-bit.
Boolean	Boolean value. Used to store boolean values (true/false).
Double	Double precision floating point value. Used to store floating point values.
Min/Max keys	Compare a value with BSON (binary JSON) element’s lowest and highest values.
Array	Used to store arrays or lists or multiple values as one key.
Timestamp	Timestamp. Records document modification or addition specific time.
Object	Used for embedded documents.
Null	Used to create empty values.
Symbol	Symbol. This data type basically equals string type, but difference is, it’s generally used for languages using special symbol types.
Date	Date time. Uses UNIX time format to store current date or time. You can specify your own date time: create Date object, pass year month day information.
Object ID	Object ID. Used to create document’s ID.
Binary Data	Binary data. Used to store binary data.
Code	Code type. Used to store JavaScript code in documents.
Regular expression	Regular expression type. Used to store regular expressions.

MongoDB - Installation and Startup

Wed, 09 May 2018 00:00:00 +0000

What is MongoDB?

MongoDB is a document-oriented free database, often used for data collection and distributed processing (Map/Reduce), especially good at big data processing.

MongoDB is a database based on distributed file storage. Written in C++ language. Aims to provide scalable high-performance data storage solutions for WEB applications.

MongoDB is a product between relational databases and non-relational databases, is the most feature-rich among non-relational databases, most like relational databases.

macOS OS Installation and Startup

Installation

Simplest is through brew installation