Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ISSUE-1847: Adding scripts and documenting the steps for neo4j backup/restore #1740

Merged
merged 1 commit into from
Nov 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions deepfence_neo4j/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
FROM neo4j:4.4.25
RUN apt update && apt install rclone -y
COPY df.sh /startup
COPY backup_neo4j.sh /usr/local/bin/
COPY load_backup_neo4j.sh /usr/local/bin/
ENTRYPOINT ["tini", "-g", "--", "/startup/df.sh"]
24 changes: 24 additions & 0 deletions deepfence_neo4j/backup_neo4j.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/bash

dt=$(date '+%Y-%m-%d_%H-%M-%S');
BACKUP_FILE="/backups/neo4j_backup_"$dt

echo "Backup file is:$BACKUP_FILE"

mkdir -p /backups/

neo4j stop
retVal=$?
if [ $retVal -ne 0 ]; then
echo "Failed to stop the neo4j db"
exit
fi

neo4j-admin dump --database='neo4j' --to=$BACKUP_FILE
retVal=$?
if [ $retVal -ne 0 ]; then
echo "Failed to create the backup file"
fi

sleep 2s
/startup/docker-entrypoint.sh neo4j >& /dev/null&
26 changes: 26 additions & 0 deletions deepfence_neo4j/load_backup_neo4j.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/bin/bash

if [ "$#" -ne 1 ]; then
echo "Missing input parameters"
echo "Correct usage:"$0" <BACKUP_FILE_FULL_PATH>"
exit
fi

BACKUP_FILE=$1
echo "Using file for db restore: $BACKUP_FILE"

neo4j stop
retVal=$?
if [ $retVal -ne 0 ]; then
echo "Failed to stop the neo4j db"
exit
fi

neo4j-admin load --from=$BACKUP_FILE --database='neo4j' --force
retVal=$?
if [ $retVal -ne 0 ]; then
echo "Failed to load the db file"
fi

/startup/docker-entrypoint.sh neo4j >& /dev/null&
sleep 2s
59 changes: 57 additions & 2 deletions docs/docs/console/database-export-import.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@
title: Database Export and Import
---

## Postgres DB Export and Import

Export PostgreSQL data from one management console and import in another console

## Export
### Export

Connect to old management console / database, run following commands to export

Expand All @@ -23,7 +25,7 @@ docker run --net=host --rm=true --name=postgresql-backup \
deepfenceio/deepfence_backup:2.0.1
```

## Import
### Import

Connect to new management console / database, run following commands to import

Expand All @@ -39,3 +41,56 @@ docker run --net=host --rm=true --name=postgresql-restore \
deepfenceio/deepfence_backup:2.0.1
```
- Restart management console once


## Neo4J Graph Database Export and Import

Export Neo4J data from one management console and Import data in another console

### Export

* Step 1: Login to the host running the neo4j docker instance.
* Step 2: Docker exec into the neo4j instance using the below command:

```shell
docker exec -it deepfence-neo4j /bin/bash
```
* Step 3: Run the backup script from inside the neo4j docker instance as follows:

```shell
/usr/local/bin/backup_neo4j.sh
```
This will create a backup file inside the container.
The name of the file will be of the format: `neo4j_backup_<YYYY-MM-DD_HOUR-MIN-SEC>`
Also, the script will print the name of the file on the stdout.
* Step 4: Copy the neo4j backup file created above to host or any intermediate location

### Import

* Step 1: Copy the backup file from intermediate location to the target machine using scp (or similar commands)
* Step 2: Login to the target machine and copy the backup file in to the running neo4j container using below command:

```shell
docker cp <BACKUP_FILE> deepfence-neo4j:/
```
* Step 3: Take a bash session of the running neo4j container using the below command:

```shell
docker exec -it deepfence-neo4j /bin/bash
```
* Step 4: Run the restore script from inside the neo4j docker instance as follows:

```shell
/usr/local/bin/load_backup_neo4j.sh /<BACKUP_FILE>
```
e.g.:
```shell
/usr/local/bin/load_backup_neo4j.sh /neo4j_backup_2023-11-17_10-25-28
```

### Steps for Kubernetes:

The steps for kubernetes remains similar to the above.
For Kubernetes, we will have to use `kubectl` utility to:
* Copy the file from and to the pod.
* Take a bash session of the pod
Loading