Wiki source code of Backup/Restore
Version 28.1 by Vincent Massol on 2020/06/29
Show last authors
| author | version | line-number | content |
|---|---|---|---|
| 1 | {{box cssClass="floatinginfobox" title="**Contents**"}} | ||
| 2 | {{toc/}} | ||
| 3 | {{/box}} | ||
| 4 | |||
| 5 | It's crucial that you define a backup strategy for your wiki so that you don't lose any data in case of a problem with your wiki installation or in case you have issues upgrading your XWiki instance to a newer version. | ||
| 6 | |||
| 7 | There are several parts that need to be backed up: | ||
| 8 | |||
| 9 | * The database | ||
| 10 | * The XWiki WAR file | ||
| 11 | * The Permanent data (located in the [[XWiki permanent directory>>platform:AdminGuide.Configuration||anchor="HConfiguringDirectories"]]) | ||
| 12 | * Log files (optional) | ||
| 13 | |||
| 14 | = Backup your database = | ||
| 15 | |||
| 16 | If you've installed XWiki using your own database then you'll need a backup tool for that database to back it up and to restore it. Moreover, if you're backing up a huge amount of data, we advise that you compress the dump files to save disk space (for example with [[7-zip>>http://www.7-zip.org/]]). | ||
| 17 | |||
| 18 | * **MySQL**:((( | ||
| 19 | Backup: {{code}}mysqldump --add-drop-database --databases xwiki > xwiki.sql{{/code}} backs up the xwiki schema (use the option ##~-~-all-databases## instead of ##~-~-databases xwiki## if you want to backup the whole XWiki istance with all its subwikis). | ||
| 20 | |||
| 21 | {{warning}} | ||
| 22 | It's important that the backup is done with ##~-~-add-drop-database## since when you restore it it's important that tables that might have been created are removed. For example database migrations can create extra tables. If you want to cancel the migration and go back to your clean dump you'll need to have those tables removed as otherwise when you run the migration again later on you might have errors. | ||
| 23 | {{/warning}} | ||
| 24 | ))) | ||
| 25 | * **PostgreSQL**: You can use {{code}}pg_dump xwiki > xwiki.sql{{/code}} to backup the PostgreSQL database. In case of backing up the whole instance, use the binary ##pg_dumpall## instead. | ||
| 26 | * **Oracle**: Use ##exp## to export data. | ||
| 27 | * **HSQLDB**: Simply backup the HSQLDB directory. For the XWiki standalone packaging it's the ##data/## directory. | ||
| 28 | |||
| 29 | = Backup the XWiki WAR = | ||
| 30 | |||
| 31 | To be on the safe side you can and should backup the whole XWiki WAR file (that's the file that was expanded and deployed in your Servlet Container when XWiki was installed). | ||
| 32 | |||
| 33 | More specifically the WAR contains mostly static files that are not changed. However it also contains some configuration files for XWiki and you really want to backup those, as explained below. | ||
| 34 | |||
| 35 | == Configuration files == | ||
| 36 | |||
| 37 | There are several configuration files you'll need to backup: | ||
| 38 | |||
| 39 | * ##WEB-INF/hibernate.cfg.xml## (Hibernate configuration) | ||
| 40 | * ##WEB-INF/xwiki.cfg## (old XWiki configuration file but still used) | ||
| 41 | * ##WEB-INF/xwiki.properties## (new XWiki Configuration file) | ||
| 42 | * ##WEB-INF/classes/logback.xml## (Logging configuration) | ||
| 43 | * ##WEB-INF/observation/*## (Cluster configuration) | ||
| 44 | |||
| 45 | To restore them simply copy them at the same location where you backed them up from. | ||
| 46 | |||
| 47 | In addition you may want to also backup any extra files you'll have installed in your XWiki installation such as JDBC drivers, extra plugins, etc. | ||
| 48 | |||
| 49 | == Custom filesystem Skins == | ||
| 50 | |||
| 51 | If you have a custom skin that is stored on the filesystem, make sure you backup the corresponding folder located in the ##webapps/xwiki/skins/## directory. | ||
| 52 | |||
| 53 | = Backup Permanent Data = | ||
| 54 | |||
| 55 | XWiki generates the following data: | ||
| 56 | |||
| 57 | * Wiki pages (but they're saved in the database) | ||
| 58 | * Attachments (when they're saved on the filesystem, otherwise they're saved in the database) | ||
| 59 | * Installed extensions | ||
| 60 | * Logs | ||
| 61 | * Mails that failed to be sent | ||
| 62 | * Cache files (Search index foles from Solr, Extension Cache, FOP Font Cache, etc) | ||
| 63 | |||
| 64 | It's important that you back them up. | ||
| 65 | |||
| 66 | All generated data is located in the [[permanent directory>>platform:AdminGuide.Configuration||anchor="HConfiguringDirectories"]] of XWiki. If you've installed the default standalone version of XWiki then the Permanent Directory is located in the ##data/## directory inside the directory where you've installed XWiki. If you have installed the XWiki WAR, you'll find this directly in the location pointed by the ##environment.permanentDirectory## configuration property in your ##WEB-INF/xwiki.properties## configuration file. | ||
| 67 | |||
| 68 | Backing up this Permanent Directory will backup all files generated by XWiki. Read on below for details. | ||
| 69 | |||
| 70 | == Attachments == | ||
| 71 | |||
| 72 | Since XWiki 3.x you can choose to [[store attachments on the filesystem>>platform:AdminGuide.Attachments||anchor="HAttachmentStorage"]] instead of in the database. If you've configured it this way then you need to backup the Permanent Directory which is where attachments saved on the filesystem are located. | ||
| 73 | |||
| 74 | == Extensions == | ||
| 75 | |||
| 76 | Extensions are also saved on the filesystem in the Permanent Data Directory in an ##extension## subdirectory. | ||
| 77 | |||
| 78 | In addition, the Extension Manager also maintains a ##cache/extension## directory that currently only contains descriptors of core extensions. If you delete it XWiki will simply try to resolve those extensions again at startup but it won't need to resolve much since the WAR embeds complete descriptors for all the JARs (since XWiki 8.4.4). Removing this directory allows cleaning descriptors that were downloaded in previous versions of XWiki. | ||
| 79 | |||
| 80 | == Solr Index Files == | ||
| 81 | |||
| 82 | If you are using Solr as the default search engine you will also have to backup the Solr index which is stored by default in the Permanent Data Directory. However XWiki will regenerate this directory automatically if it's not found at startup. The Solr index files are stored in ##cache/solr/search## (since XWiki 12.5, they used to be in ##solr/## from XWiki 4.3 to 12.4). | ||
| 83 | |||
| 84 | == Mails == | ||
| 85 | |||
| 86 | The [[Mail Sender API>>extensions:Extension.Mail Sender API]] (used by several features of XWiki that require sending mails) will serialize email messages in a ##mails## subdirectpry inside the permanent directory before they are sent, so that if XWiki crashes or is stopped, those mails are not lost and can be resent. Once a mail is successfully sent, its associated file is removed from the ##mails## directory. | ||
| 87 | |||
| 88 | = Backup Log files = | ||
| 89 | |||
| 90 | XWiki generates logs in the console by default and these will go where your Servlet container has been defined to save them. For Tomcat for example this usually goes in the ##TOMCAT_HOME/logs/catalina.out## file. | ||
| 91 | |||
| 92 | However this is fully [[configurable>>platform:AdminGuide.Logging]] and you may have configured XWiki to output log files elsewhere. If you want to keep those logs, don't forget to save them. Those logs are not needed by XWiki but you may want to save them to review them later on to diagnose issues that happened for example. | ||
| 93 | |||
| 94 | = Using the XWiki Export feature = | ||
| 95 | |||
| 96 | Since most XWiki data is saved in wiki pages one solution to backup a XWiki instance is to export all its pages using XWiki's [[Import/Export feature>>platform:AdminGuide.ImportExport]]. However note that this is quite a resource hungry operation and depending on the size of your wiki you may need a lot of memory. As a consequence the recommended backup strategy is to backup the databases. It's much better to use a specialized backup tool that'll backup the database, perform incremental backups, verify backup integrity, etc. | ||
| 97 | |||
| 98 | In addition you should know that currently the following information is not located in wiki pages and is thus not saved in a wiki export: | ||
| 99 | |||
| 100 | * Activity stream data | ||
| 101 | * Statistics data | ||
| 102 | * Feed plugin data (if you use it) | ||
| 103 | * Deleted documents/attachments data | ||
| 104 | * XWiki Instance Id | ||
| 105 | * Mail Statuses | ||
| 106 | |||
| 107 | Moreover this won't save configuration data or other Permanent Data mentioned above such as installed Extensions, Lucene/SOLR index files and Logs. | ||
| 108 | |||
| 109 | In practice the Import/Export feature should be reserved for the following use cases: | ||
| 110 | |||
| 111 | * move data across XWiki instances, including sharing of applications between separate instances | ||
| 112 | * move data to another wiki | ||
| 113 | * convert from one database to another | ||
| 114 | |||
| 115 | = Restore your data = | ||
| 116 | |||
| 117 | * **MySQL**: Assuming you have your ##xwiki.sql## file handy (obtained running ##mysqldump## as explained above), run the following from the command line (or a script). Examples below are for the Linux shell.((( | ||
| 118 | * Disable constraints check first: {{code}}mysql -e "SET FOREIGN_KEY_CHECKS=0;"{{/code}} (restoring may fail with some cryptic error code, leaving the database in an inconsistent state, if you omit this step) | ||
| 119 | * Delete the existing ##xwiki## database if any: {{code}}mysql -e "DROP DATABASE xwiki;"{{/code}} (better to start fresh) | ||
| 120 | * Recreate the ##xwiki## database: {{code}}mysql -e "CREATE DATABASE IF NOT EXISTS xwiki DEFAULT CHARACTER SET utf8;"{{/code}}. | ||
| 121 | * Restore data from the dump file: {{code}}mysql xwiki --user=root -p < xwiki.sql{{/code}}. Alternatively, you can omit the ##-p## by adding a section [mysqldump] in your ##.my.cnf## file (with a user and password line, same as for [mysql]). | ||
| 122 | * Re-enable constraints checks: {{code}}mysql -e "SET FOREIGN_KEY_CHECKS=1;"{{/code}} | ||
| 123 | |||
| 124 | That is not all, now you need to restore the xwiki user rights (assuming the ##xwiki## user exists in MySQL). | ||
| 125 | |||
| 126 | * Update the xwiki user password from the (previously restored) hibernate configuration file((( | ||
| 127 | {{code language="none"}} | ||
| 128 | pass=$(cat /etc/xwiki/hibernate.cfg.xml | grep passw| head -1 | sed -e 's/^.*<prop.*password">//' | sed -e 's|</property>.*||') | ||
| 129 | mysql -e "SET PASSWORD FOR 'xwiki'@'localhost' = PASSWORD('$pass');" | ||
| 130 | {{/code}} | ||
| 131 | ))) | ||
| 132 | |||
| 133 | Finally, restart you container, for example in Linux: {{code}}service tomcat7 restart{{/code}} | ||
| 134 | ))) | ||
| 135 | * **PostgreSQL**: {{code}}psql -d xwiki -f xwiki.sql{{/code}} to restore your previously saved data. In full instance backup mode, use {{code}}psql -d postgres -f xwiki.sql{{/code}} instead. | ||
| 136 | * **Oracle**: Use ##imp##. | ||
| 137 | |||
| 138 | = Tips in case you are using nginx = | ||
| 139 | |||
| 140 | Assuming you have a nginx configuration with a {{code}}server_name{{/code}} entry that needs updating, the following script might be useful. | ||
| 141 | |||
| 142 | {{code}} | ||
| 143 | #!/bin/bash | ||
| 144 | configfile="/etc/nginx/sites-available/xwiki-as-root.conf" | ||
| 145 | # The following line works for Amazon EC2 instances. Change accordingly to find this host's IP dynamically. | ||
| 146 | thisip=$(curl -s http://instance-data/latest/meta-data/public-ipv4) | ||
| 147 | if [ ! -f $configfile ]; then | ||
| 148 | echo "Missing file: $configfile" | ||
| 149 | exit 1 | ||
| 150 | fi | ||
| 151 | echo backing-up $configfile | ||
| 152 | cp $configfile $configfile-old | ||
| 153 | |||
| 154 | echo Setting nginx server_name to $thisip | ||
| 155 | old=$(cat $configfile | grep '^\ *server_name' | awk '{print $2}' | sed -e 's/;//' | grep "$thipip" | head -1) | ||
| 156 | if [ $? -eq 0 ] ; then | ||
| 157 | echo Fixing server_name | ||
| 158 | cat $configfile | sed -e "s/$old/$thisip/" > $configfile-new | ||
| 159 | mv $configfile-new $configfile | ||
| 160 | fi | ||
| 161 | |||
| 162 | echo Done. New config file: | ||
| 163 | cat $configfile | ||
| 164 | |||
| 165 | echo restarting nginx | ||
| 166 | service nginx restart | ||
| 167 | {{/code}} | ||
| 168 | |||
| 169 | = Sample Backup Script = | ||
| 170 | |||
| 171 | {{code}} | ||
| 172 | #!/bin/bash | ||
| 173 | |||
| 174 | #Space Separated List of Databases to Dump | ||
| 175 | #DATABASE="xwiki d1 d3" | ||
| 176 | DATABASE="xwiki" | ||
| 177 | DBUSER=root | ||
| 178 | DBPASS=***** | ||
| 179 | |||
| 180 | #XWIKI data folder | ||
| 181 | DATAFOLDER=/srv/tomcat6/data/ | ||
| 182 | #Where is the webapps folder for your tomcat installation | ||
| 183 | # SLES 11 is located /srv/tomcat6/webapps | ||
| 184 | WEBAPPDIR=/srv/tomcat6/webapps | ||
| 185 | #What context (dir) does your application deploy to | ||
| 186 | DEPLOYCONTEXT=ROOT | ||
| 187 | |||
| 188 | |||
| 189 | ###~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
| 190 | DEPLOYDIR=${WEBAPPDIR}/${DEPLOYCONTEXT} | ||
| 191 | DATE=$(date '+%Y-%m-%d') | ||
| 192 | mkdir ./${DATE} | ||
| 193 | |||
| 194 | #backup mysql | ||
| 195 | echo "Backing up Mysql" | ||
| 196 | mysqldump -h 127.0.0.1 -u ${DBUSER} --password=${DBPASS} --max_allowed_packet=512m --add-drop-database --databases ${DATABASE} | /bin/gzip > ./${DATE}/${DATABASE}.sql.gz | ||
| 197 | |||
| 198 | echo "Backing up Data" | ||
| 199 | #Backup Exteral Data Storage | ||
| 200 | /bin/tar -C ${DATAFOLDER}/../ -zcf ./${DATE}/data.tar.gz data | ||
| 201 | |||
| 202 | #Backing Java Keystore | ||
| 203 | /bin/cp /srv/tomcat6/.keystore ./${DATE}/.keystore | ||
| 204 | |||
| 205 | echo "Backing up xwiki configuration" | ||
| 206 | /bin/cp ${DEPLOYDIR}/WEB-INF/hibernate.cfg.xml ./${DATE}/hibernate.cfg.xml | ||
| 207 | /bin/cp ${DEPLOYDIR}/WEB-INF/xwiki.cfg ./${DATE}/xwiki.cfg | ||
| 208 | /bin/cp ${DEPLOYDIR}/WEB-INF/xwiki.properties ./${DATE}/xwiki.properties | ||
| 209 | |||
| 210 | #Backup Deploy Context | ||
| 211 | echo "Backing UP deploy Context" | ||
| 212 | /bin/tar -C ${DEPLOYDIR}/../ -zcf ./${DATE}/ROOT.tar.gz ROOT | ||
| 213 | |||
| 214 | echo "DONE" | ||
| 215 | {{/code}} |
