pt-online-schema-change VS gh-ost

Blocking Use Cases of pt-online-schema-change:

  • It is not possible to reduce the overhead of the tool to 0. Pt-online-schema-change gives you an option to define the maximum allowed replication lag and, if that threshold is crossed, it stops to copy data between the old and new table. It is also possible to pause the background process entirely. The problem is that we are talking only about the background process of running INSERTs. It is not possible to reduce the overhead caused by the fact that every operation in “yourtable” is duplicated in “yourtable_new” through triggers. If you remove the triggers, the old and new table would go out of sync without any means to sync them again. Therefore, when you run pt-online-schema-change on your system, it always adds some overhead, even if it is paused or throttled. How big overhead depends on how many writes hit the table which is undergoing a schema change.
  • Another issue is caused again by triggers – precisely by the fact that, to create triggers, one has to acquire a lock on MySQL’s metadata. This can become a serious problem if you have highly concurrent traffic or if you use longer transactions. Under such load, it may be virtually impossible to use pt-online-schema-change due to the fact that it is not able to acquire metadata lock to create the required triggers. Additionally, the process of acquiring metadata can also lock further transactions, basically grinding all database operations to halt.
  • Rename table requires lock on table any long transcation or high load keep rename table operation in waiting for mtadata lock, which cause further transactions to be on hold till completion of rename table, which may cause max connection limit reached in database.
  • Yet another problem are foreign keys – unfortunately, there is no simple way of handling them. Pt-online-schema-change gives you two methods to approach this issue. Neither of those are really good. The main issue here is that a foreign key of a given name can only refer to a single table and it sticks to it – even if you rename the table referred to, the foreign key will follow this change. This leads to the problem: after RENAME TABLE, the foreign key will point to ‘yourtable_old’, not ‘yourtable’.

 

Blocking Use Cases of gho-ost:

  • Does not support SBR(Statment based Replication) or Mixed, Requires atleast one slave with RBR(Row Based Replication).
  • Does not Support Foreign Keys;
  • Does not Support tables schema change with triggers.
  • Does not support Galera Cluster.
  • Minimal row image is not supported (which makes your binlogs grow larger), JSON and generated columns in 5.7 are not supported.
  • Network traffic is increased compared to pt-online-schema-change – not only gh-ost has to copy data but it also has to copy binary logs.

 

Benefits of gho-ost:

  • You case stop 100% load of schema change process.

 

Supporting Links:

https://severalnines.com/blog/online-schema-change-mysql-mariadb-comparing-github-s-gh-ost-vs-pt-online-schema-change

renamecollection in initial sync mongodb

renamecollection or any action using renamecollection like aggregation in oplog drop all data on node in startup2 mode and restarts initial sync from mongo version 3.2.12 and this is up to all versions in 3.4. issue has fix in version 3.6

Related Jira :
https://jira.mongodb.org/browse/SERVER-38524
https://jira.mongodb.org/browse/SERVER-4941

Logs in mongo error log..

Tue Dec 11 11:16:42.601 E REPL [repl writer worker 9] Error applying command ({ ts: Timestamp 1544373391000|135, t: 17, h: -4526450696875989552, v: 2, op: “c”, ns: “xdb.$cmd”, o: { renameCollection: “xdb.tmp.agg_out.136”, to: “xdb.xcollection”, stayTemp: false, dropTarget: true } }): OplogOperationUnsupported: Applying renameCollection not supported in initial sync: { ts: Timestamp 1544373391000|135, t: 17, h: -4526450696875989552, v: 2, op: “c”, ns: “xdb.$cmd”, o: { renameCollection: “xdb.t
mp.agg_out.136”, to: “xdb.xcollection”, stayTemp: false, dropTarget: true } }
Tue Dec 11 11:16:42.601 E REPL [repl writer worker 9] Error applying command ({ ts: Timestamp 1544373391000|135, t: 17, h: -4526450696875989552, v: 2, op: “c”, ns: “xdb.$cmd”, o: { renameCollection: “xdb.tmp.agg_out.136”, to: “xdb.xcollection”, stayTemp: false, dropTarget: true } }): OplogOperationUnsupported: Applying renameCollection not supported in initial sync: { ts: Timestamp 1544373391000|135, t: 17, h: -4526450696875989552, v: 2, op: “c”, ns: “xdb.$cmd”, o: { renameCollection: “xdb.t
mp.agg_out.136”, to: “xdb.xcollection”, stayTemp: false, dropTarget: true } }
Tue Dec 11 11:16:42.601 E REPL [replication-221] Failed to apply batch due to ‘OplogOperationUnsupported: error applying batch: Applying renameCollection not supported in initial sync: { ts: Timestamp 1544373391000|135, t: 17, h: -4526450696875989552, v: 2, op: “c”, ns: “xdb.$cmd”, o: { renameCollection: “xdb.tmp.agg_out.136”, to: “xdb.xcollection”, stayTemp: false, dropTarget: true } }’
Tue Dec 11 11:16:42.601 I REPL [replication-223] Finished fetching oplog during initial sync: CallbackCanceled: Callback canceled. Last fetched optime and hash: { ts: Timestamp 1544507199000|3, t: 18 }[-5401731233052630226]
Tue Dec 11 11:16:42.602 E REPL [replication-223] Initial sync attempt failed — attempts left: 7 cause: OplogOperationUnsupported: error applying batch: Applying renameCollection not supported in initial sync: { ts: Timestamp 1544373391000|135, t: 17, h: -4526450696875989552, v: 2, op: “c”, ns: “xdb.$cmd”, o:{ renameCollection: “xdb.tmp.agg_out.136”, to: “xdb.xcollection”, stayTemp: false, dropTarget: true } }
Tue Dec 11 11:16:43.602 I REPL [replication-222] Starting initial sync (attempt 4 of 10)
Tue Dec 11 11:16:43.620 I REPL [replication-222] sync source candidate: 172.29.126.139:27017
Tue Dec 11 11:16:43.689 I STORAGE [replication-222] dropAllDatabasesExceptLocal 4
Tue Dec 11 11:16:50.019 I REPL [replication-222] ******
Tue Dec 11 11:16:50.019 I REPL [replication-222] ******