Index WP MySQL For Speed

Description

How do I use this plugin?

After you install and activate this plugin, visit the Index MySQL Tool under the Tools menu. From there you can press the Add Keys Now button. If you have large tables, use it with WP-CLI instead to avoid timeouts. See the WP-CLI section to learn more.

What does it do for my site?

This plugin works to make your MySQL database work more efficiently by adding high-performance keys to the tables you choose. On request it monitors your site’s use of your MySQL database to detect which database operations are slowest. It is most useful for large sites: sites with many users, posts, pages, and / or products.

You can use it to restore WordPress’s default keys if need be.

What is this all about?

Where does WordPress store all that stuff that makes your site great? Where are your pages, posts, products, media, users, custom fields, metadata, and all your valuable content? All that data is in the MySQL relational database management system. (Many hosting providers and servers use the MariaDB fork of the MySQL software; it works exactly the same way as MySQL itself.)

As your site grows, your MySQL tables grow. Giant tables can make your page loads slow down, frustrate your users, and even hurt your search-engine rankings. And, bulk imports can take absurd amounts of time. What can you do about this?

You can install and use a database cleaner plugin to get rid of old unwanted data and reorganize your tables. That makes them smaller, and therefore faster. That is a good and necessary task. That is not the task of this plugin. You can, if your hosting provider supports it, install and use a Persistent Object Cache plugin to reduce traffic to your database. That is not the task of this plugin either.

This plugin adds database keys (also called indexes) to your MySQL tables to make it easier for WordPress to find the information it needs. All relational database management systems store your information in long-lived tables. For example, WordPress stores your posts and other content in a table called wp_posts, and custom post fields in another table called wp_postmeta. A successful site can have thousands of posts and hundreds of thousands of custom post fields. MySQL has two jobs:

  1. Keep all that data organized.
  2. Find the data it needs quickly.

To do its second job, MySQL uses database keys. Each table has one or more keys. For example, wp_posts has a key to let it quickly find posts when you know the author. Without its post_author key MySQL would have to scan every one of your posts looking for matches to the author you want. Our users know what that looks like: slow. With the key, MySQL can jump right to the matching posts.

In a new WordPress site with a couple of users and a dozen posts, the keys don’t matter very much. As the site grows the keys start to matter, a lot. Database management systems are designed to have their keys updated, adjusted, and tweaked as their tables grow. They’re designed to allow the keys to evolve without changing the content of the underlying tables. In organizations with large databases adding, dropping, or altering keys doesn’t change the underlying data. It is a routine maintenance task in many data centers. If changing keys caused databases to lose data, the MySQL and MariaDB developers would hear howling not just from you and me, but from many heavyweight users. (You should still back up your WordPress instance of course.)

Better keys allow WordPress’s code to run faster without any code changes. Experience with large sites shows that many MySQL slowdowns can be improved by better keys. Code is poetry, data is treasure, and database keys are grease that makes code and data work together smoothly.

Which tables does the plugin add keys to?

This plugin adds and updates keys in these WordPress and WooCommerce tables.

  • wp_comments
  • wp_commentmeta
  • wp_posts
  • wp_postmeta
  • wp_termmeta
  • wp_users
  • wp_usermeta
  • wp_options
  • wp_wc_orders_meta
  • wp_woocommerce_order_itemmeta
  • wp_automatewoo_log_meta

You only need run this plugin once to get its benefits.

How can I monitor my database’s operation?

On the Index MySQL page (from your Tools menu on your dashboard), you will find the “Monitor Database Operations” tab. Use it to request monitoring for a number of minutes you choose.

You can monitor

  • either the site (your user-visible pages) or the dashboard, or both.
  • all pageviews, or a random sample. (Random samples are useful on very busy sites to reduce monitoring overhead.)

Once you have gathered monitoring information, you can view the captured queries, and sort them by how long they take. Or you can save the monitor information to a file and show it to somebody who knows about database operations. Or you can upload the monitor to the plugin’s servers so the authors can look at it.

It’s a good idea to monitor for a five-minute interval at a time of day when your site is busy. Once you’ve completed a monitor, you can examine it to determine which database operations are slowing you down the most.

Please consider uploading your saved monitors to the plugin’s servers. It’s how we learn from your experience to keep improving. Push the Upload button on the monitor’s tab.

WP-CLI command line operation

This plugin supports WP-CLI. When your tables are large this is the best way to add the high-performance keys: it doesn’t time out.

Give the command wp help index-mysql for details. A few examples:

  • wp index-mysql status shows the current status of high-performance keys.
  • wp index-mysql enable --all adds the high-performance keys to all tables that don’t have them.
  • wp index-mysql enable wp_postmeta adds the high-performance keys to the postmeta table.
  • wp index-mysql disable --all removes the high-performance keys from all tables that have them, restoring WordPress’s default keys.
  • wp index-mysql enable --all --dryrun writes out the SQL statements necessary to add the high-performance keys to all tables, but does not run them.
  • wp index-mysql enable --all --dryrun | wp db query writes out the SQL statements and pipes them to wp db to run them.

Note: avoid saving the –dryrun output statements to run later. The plugin generates them to match the current state of your tables.

Why use this plugin?

Three reasons (maybe four):

  1. to save carbon footprint.
  2. to save carbon footprint.
  3. to save carbon footprint.
  4. to save people time.

Seriously, the microwatt hours of electricity saved by faster web site technologies add up fast, especially at WordPress’s global scale.

How can I learn more about making my WordPress site more efficient?

We offer several plugins to help with your site’s database efficiency. You can read about them here.

Credits

  • Michael Uno for Admin Page Framework.
  • Marco Cesarato for LiteSQLParser.
  • Allan Jardine for Datatables.net.
  • Leho Kraav and Sebastian Sommer for suggesting the WooCommerce tables.
  • Japreet Sethi for advice, and for testing on his large installation.
  • Rick James for everything.
  • Jetbrains for their IDE tools, especially PhpStorm. It’s hard to imagine trying to navigate an epic code base without their tools.

Screenshots

  • Use Tools > Index MySQL to view the Dashboard panel.
  • Choose tables and add High-Performance Keys.
  • Start Monitoring Database Operations, and see saved monitors.
  • View a saved monitor to see slow database queries.
  • About the plugin.
  • Use WP CLI to run the plugin’s operations.

Installation

You may install this plugin by visiting Plugins > Add New on your site’s Dashboard, then searching for Index WP MySQL For Speed and following the usual installation workflow.

When you activate it, it will copy a php source file into the must-use plugins directory, wp-content/mu-plugins. Some sites’ configurations prevent the web server from writing files into that directory. In that case the plugin will still work correctly. But, after WordPress core version upgrades you may have to revisit the Tools > Index MySQL page and correct the keying on some tables. Why? The mu-plugin prevents core version updates from trying to change keys.

Composer

If you configure your WordPress installation using composer, you may install this plugin into your WordPress top level configuration with the command

composer require "wpackagist-plugin/index-wp-mysql-for-speed":"^1.4"

During composer installation the plugin can automatically copy the necessary source file (see the previous section) into the must-use plugins directory. If you want that to happen, you should include these scripts in your top-level composer.json file.

 "scripts": {
         "install-wp-mysql-mu-module": [
                 "@composer --working-dir=wordpress/wp-content/plugins/index-wp-mysql-for-speed install-mu-module"
         ],
         "post-install-cmd": [
                 "@install-wp-mysql-mu-module"
         ],
         "post-update-cmd": [
                 "@install-wp-mysql-mu-module"
         ]
     },

FAQ

Should I back up my site before using this?

Yes. You already knew that.

I don’t see any changes to my database speed. Why not?

  • Just installing and activating the plugin is not enough to make it work. Don’t forget to visit the Index MySQL Tool under the Tools menu. From there you can press the Add Keys Now button.
  • On a modestly sized site (with a few users and a few hundred posts) your database may be fast enough without these keys. The speed improvements are most noticeable on larger sites with many posts and products.

I use a nonstandard database table prefix. Will this work ?

Yes. Some WordPress databases have nonstandard prefixes. That is, their tables are named something_posts, something_postmeta, and so forth instead of wp_posts and wp_postmeta. This works with those databases.

My WordPress host offers MariaDB, not MySQL. Can I use this plugin?

Yes.

Which versions of MySQL and MariaDB does this support?

MySQL versions 5.5.62 and above, 5.6.4 and above, 8 and above. MariaDB versions 5.5.62 and above.

What database Storage Engine does this support?

InnoDB only. If your tables use MyISAM (the older storage engine) or the older COMPACT row format, this plugin offers to upgrade them for you.

What tables and keys does the plugin change?

Please read this.

Is this safe? Can I add high-performance keys and revert back to WordPress standard keys safely?

Yes. it is safe to add keys and revert them. Changing keys is a routine database-maintenance operation.

As you know you should still keep backups of your site: other things can cause data loss.

My site uses WooCommerce HPOS (High Performance Order Storage). Is this plugin still helpful?

Yes. WooCommerce still uses core WordPress tables for your shop’s products, posts, pages, and users. This plugin adds high-performance keys to those tables.

High Performance Order Storage, true to its name, stores your shop’s orders in a more efficient way. Formerly orders were stored in those same core WordPress tables.

Is this plugin compatible with some other specific plugin?

This plugin only changes database indexes. If the other plugin does not change database indexes, it is very likely compatible with this one.

Of course, if you find an incompatibility please open a support topic.

I got a fatal error trying to add keys. How can I fix that?

Sometimes the Index WP MySQL For Speed plugin for WordPress generates errors when you use it to add keys. These can look like this or similar:

Fatal error: Uncaught ImfsException: [0]: Index for table 'wp_postmeta' is corrupt; try to repair it

First, don’t panic. This (usually) does not mean your site has been corrupted. It simply means your MariaDB or MySQL server was not able to add the keys to that particular table. Your site will still function, but you won’t get the benefit of high-performance keys on the particular table. Very large tables are usually the ones causing this kind of error. Very likely you ran out of temporary disk space on your MariaDB or MySQL database server machine. The database server makes a temporary copy of a table when you add keys to it; that allows it to add the keys without blocking your users.

It’s possible to correct this problem by changing your MariaDB or MySQL configuration. Instructions are here.

What happens to my tables and keys during a WordPress version update?

If the plugin is activated during a WordPress version update, it prevents the update workflow from removing your high-performance keys (Version 1.4.7).

My site has thousands of registered users. My Users, Posts, and Pages panels in my dashboard are still load slowly even with this plugin.

We have another plugin to handle lots of users, Index WP Users For Speed. Due to the way WordPress handles users, just changing database keys is not enough to solve those performance problems.

How can I enable persistent object caching on my site?

Persistent object caching can help your site’s database performance by reducing its workload. You can read about it here. If your hosting provider doesn’t offer redis or memcached cache-server technology you can try using our SQLite Object Cache plugin for the purpose.

Why did the size of my tables grow when I added high-performance keys?

Database keying works by making copies of your table’s data organized in ways that are easy to randomly access. Your MariaDB or MySQL server automatically maintains the copies of your data as you insert or update rows to each table. And, the keying task adjusts the amount of free space in each block of your table’s data in preparation for the insertion of new rows. When free space is available, inserting new rows doesn’t require relatively slow block splits. Tables that have been in use for a long time often need new free space in many blocks. When adding keys, it is normal for table sizes to increase. It’s the oldest tradeoff in computer science: time vs. space.

Will the new keys be valid for new data in the tables?

Yes. Once the high-performance keys are in place MariaDB and MySQL automatically maintain them as you update, delete, or insert rows of data to your tables. There is no need to do anything to apply the keys to new data: the DBMS software does that for you.

How do I revert to WordPress’s standard keys, undoing the action of this plugin?

You can revert the keys from the Index MySQL Tool under the Tools menu, or use the wp-cli command wp index-mysql disable --all. Notice that if you deactivate or delete the plugin without doing this, the high-performance keys remain.

How do I get an answer to another question?

Please see more questions and answers here.

Reviews

24, Payares de 2024 1 reply
It was days that I had problems with mariadb that every time I opened the homepage or any article it started to go 100/200% cpu, it became unmanageable, I tried a lot of solutions but nothing was solving, a friend found me this plugin that solved (at least from the first 4 hours it didn’t happen anymore) the problem.Thank you very much.
2, Ochobre de 2024
We have been using this plugin since its initial release, their updates are brilliant never a hitch and being a fast based ecommerce company we need all the speed we can get. Their most recent update covering post_meta tables is a welcomed addition to an already underrated plugin that should be built into core. We will continue to use these plugins on all our larger installs.
12, Setiembre de 2024 4 replies
As a heavy user and someone familiar with the core issues of WordPress, I believe the lack of proper indexing is a major oversight, particularly noticeable on larger sites. This plugin, “Index WP MySQL For Speed”, addresses this by offering potential CPU savings through efficient indexing. While it doesn’t miraculously speed up your site, it does optimize resource use. In my case, with over 600,000 orders and daily transactions ranging between 1,000 to 2,000 outside of peak times, I couldn’t conclusively measure significant performance improvements in my personal monitoring graphs. However, I did notice no degradation in performance and possibly saved CPU usage. The real magic, however, lies in the configuration of your MariaDB. Our system is finely tuned and runs locally to minimize latency, which is critical. Thus, I recommend this plugin for medium to large databases, especially for those with database and sysadmin expertise. Please ensure your site and database are optimized first. This plugin won’t speed up a site that is inherently slow; it makes it more efficient. However, I would not recommend this plugin for someone on shared hosting without the necessary resources and tools. Remember, utilizing WP-CLI is advisable for any operations that require database modifications. This is from experience; if you want to save time and be proactive, this is the way to go. Config Details: MariaDB 11.4, Customized for Our Needs. System: LEMP + Redis, 64 GB RAM. Database Size: 25 GB. A big thank you to the developers for their hard work and dedication in creating such a valuable tool!
3, Setiembre de 2024
I’ve had very slow wp-admin pages for a while now. Query monitor plugin showed a very high number of database queries from the woocommerce plugin. No idea why. Installing this plugin and Fast Woo Order Lookup plugin has immediately reduced the very high number of queries and sped up both wp-admin and woocommerce admin pages. Thank you very much and I hope my admin stays working this quick !
15, Agostu de 2024
Really genius work!!”Code is poetry, data is treasure, and database keys are grease that makes code and data work together smoothly.”
4, Xunetu de 2024 2 replies
This is an excellent plugin. @olliejones, thank you for contributing this valuable resource to the community. We drew inspiration from this plugin and developed a custom solution by incorporating elements from it. I have a minor suggestion: while this plugin is powerful, it should be implemented with care. Adding a brief note advising users to test the plugin in a DEV/TEST environment before deploying it in production could help prevent potential issues like crashes. However, we did not encounter any problems, and everything worked smoothly. Lastly, a quick question for @olliejones: Are there specific reasons why WordPress does not include such optimized index keys within its core?
Read all 82 reviews

Contributors & Developers

“Index WP MySQL For Speed” is open source software. The following people have contributed to this plugin.

Contributors

“Index WP MySQL For Speed” has been translated into 2 locales. Thank you to the translators for their contributions.

Translate “Index WP MySQL For Speed” into your language.

Interested in development?

Browse the code, check out the SVN repository, or subscribe to the development log by RSS.

Changelog

1.5.2

Correct release error.

1.5.1

Add support for the WooCommerce tables wp_automatewoo_log_meta, wp_wc_orders_meta, and wp_woocommerce_order_itemmeta.

1.4.19

Report information about host machine using meminfo and procinfo if those files are available.

1.4.18

Security update.

1.4.17

Back out a miscellaneous bug fix from the previous version. It was an attempt to avoid a warning from Query Monitor’s hooks display.
Upload the full MariaDB / MySQL version information with monitors as well as metadata.

1.4.16

(no changes to keys)
WordPress 6.5 compatibility.
Support WordPress versions back to 4.2 (At MDDHosting’s request).
Avoid attempting to upgrade from storage engines except MyISAM and Aria.
WP-CLI upgrade, enable, and disable commands are idempotent now. They don’t generate errors when they find no tables to process.
Miscellaneous bug fixes