Switch to MySQL 8 #103

kdambekalns · 2024-10-11T15:24:12Z

With this Local Beach uses MySQL 8 instead of MariaDB 10 to be closer to the setup used in Beach.

Running beach setup will migrate all databases from MariaDB to MySQL if the path used previously exists.

With this Local Beach uses MySQL 8 instead of MariaDB 10 to be closer to the setup used in Beach. Running `beach setup` will migrate all databases from MariaDB to MySQL if the path used previously exists.

kdambekalns · 2025-04-02T15:36:53Z

Test run result:

$ beach setup
DEBUG setting up Local Beach with base path /Users/karsten/.LocalBeach
DEBUG creating directory for certificates at /Users/karsten/.LocalBeach/Certificates
DEBUG creating directory for databases at /Users/karsten/.LocalBeach/MySQL
INFO Migrating MariaDB data from /Users/karsten/.LocalBeach/MariaDB to MySQL at /Users/karsten/.LocalBeach/MySQL
WARNING Note: This may take a while, depending on DB size!
DEBUG starting MariaDB server ...
[+] Running 2/2
 ✔ Container local_beach_database  Started                                                                                                                                       ✔ Container local_beach_mariadb   Started                                                                                                                                     
DEBUG waiting for MariaDB server ...
DEBUG waiting for MySQL server ...
DEBUG dumping data from MariaDB to MySQL
DEBUG … acmecom
DEBUG … beach-cluster-agent
DEBUG … beach-controlpanel
DEBUG … docsneosio
DEBUG … flownativecom
DEBUG … idneosio
DEBUG … neos-demo
DEBUG … neosio
DEBUG stopping MariaDB server ...
INFO Done with migration to MySQL at /Users/karsten/.LocalBeach/MySQL
INFO If all works as expected, remove /Users/karsten/.LocalBeach/MariaDB

kdambekalns · 2025-04-02T15:41:30Z

When testing, build this, run beach setup (using the freshly built binary!) and then beach start (using the freshly built binary!) again. Everything should work as before. 🤞

robertlemke · 2025-04-02T16:23:41Z

DEBUG setting up Local Beach with base path /Users/robert/.LocalBeach 
DEBUG creating directory for certificates at /Users/robert/.LocalBeach/Certificates 
DEBUG creating directory for databases at /Users/robert/.LocalBeach/MySQL 
INFO Migrating MariaDB data from /Users/robert/.LocalBeach/MariaDB to MySQL at /Users/robert/.LocalBeach/MySQL 
WARNING Note: This may take a while, depending on DB size! 
DEBUG starting MariaDB server ...                  
panic: open /local-beach/mariadb-compose.yml: file does not exist

goroutine 1 [running]:
github.com/flownative/localbeach/cmd/beach/cmd.readFileFromAssets({0x10335aac0?, 0x102826f04?})
        /Users/robert/Projects/localbeach/cmd/beach/cmd/helpers.go:72 +0x1c0
github.com/flownative/localbeach/cmd/beach/cmd.writeMariaDBComposeFile()
        /Users/robert/Projects/localbeach/cmd/beach/cmd/helpers.go:166 +0x2c
github.com/flownative/localbeach/cmd/beach/cmd.startMariaDB()
        /Users/robert/Projects/localbeach/cmd/beach/cmd/setup.go:177 +0x58
github.com/flownative/localbeach/cmd/beach/cmd.migrateMariaDBToMySQL()
        /Users/robert/Projects/localbeach/cmd/beach/cmd/setup.go:140 +0xe4
github.com/flownative/localbeach/cmd/beach/cmd.setupLocalBeach()
        /Users/robert/Projects/localbeach/cmd/beach/cmd/setup.go:124 +0x27c
github.com/flownative/localbeach/cmd/beach/cmd.handleSetupRun(0x140002abc00?, {0x104494ee0?, 0x4?, 0x10333a6a7?})
        /Users/robert/Projects/localbeach/cmd/beach/cmd/setup.go:43 +0x1c
github.com/spf13/cobra.(*Command).execute(0x1044598e0, {0x104494ee0, 0x0, 0x0})
        /Users/robert/go/pkg/mod/github.com/spf13/[email protected]/command.go:1019 +0x810
github.com/spf13/cobra.(*Command).ExecuteC(0x104459360)
        /Users/robert/go/pkg/mod/github.com/spf13/[email protected]/command.go:1148 +0x350
github.com/spf13/cobra.(*Command).Execute(...)
        /Users/robert/go/pkg/mod/github.com/spf13/[email protected]/command.go:1071
github.com/flownative/localbeach/cmd/beach/cmd.Execute()
        /Users/robert/Projects/localbeach/cmd/beach/cmd/root.go:33 +0x24
main.main()
        /Users/robert/Projects/localbeach/main.go:8 +0x1c

I'll check what's going wrong here …

robertlemke · 2025-04-02T17:38:12Z

Sorry, I didn't compile with make and therefore forgot to re-generate the assets.

The setup run worked now:

DEBUG setting up Local Beach with base path /Users/robert/.LocalBeach 
DEBUG creating directory for certificates at /Users/robert/.LocalBeach/Certificates 
DEBUG creating directory for databases at /Users/robert/.LocalBeach/MySQL 
INFO Migrating MariaDB data from /Users/robert/.LocalBeach/MariaDB to MySQL at /Users/robert/.LocalBeach/MySQL 
WARNING Note: This may take a while, depending on DB size! 
DEBUG starting MariaDB server ...                  
INFO Reading file from internal assets: local-beach/mariadb-compose.yml 
[+] Running 12/12
 ✔ database Pulled                                                                                                                        26.2s 
   ✔ fa07288abb8b Pull complete                                                                                                            6.3s 
   ✔ 0a1d246ada2d Pull complete                                                                                                            6.3s 
   ✔ 403da7fd01e2 Pull complete                                                                                                            6.4s 
   ✔ 8c0d86ac38d6 Pull complete                                                                                                            6.6s 
   ✔ 25df6f366614 Pull complete                                                                                                            6.6s 
   ✔ 3af997c5168a Pull complete                                                                                                            6.6s 
   ✔ 92c4da0fef3d Pull complete                                                                                                           10.1s 
   ✔ 6940dbb4eace Pull complete                                                                                                           10.1s 
   ✔ 15896173647e Pull complete                                                                                                           23.2s 
   ✔ fb5d63cb0124 Pull complete                                                                                                           23.2s 
   ✔ d5bda411c15d Pull complete                                                                                                           23.2s 
WARN[0026] Found orphan containers ([local_beach_nginx]) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up. 
[+] Running 2/2
 ✔ Container local_beach_mariadb   Started                                                                                                 2.3s 
 ✔ Container local_beach_database  Started                                                                                                 2.2s 
DEBUG waiting for MariaDB server ...               
DEBUG waiting for MySQL server ...                 
DEBUG dumping data from MariaDB to MySQL           
DEBUG … 0jtkcbfokj8u5eze                           
DEBUG … beach-cluster-agent                        
DEBUG … beach-controlpanel                         
DEBUG … beach-controlpanel-debug                   
DEBUG … beach-controlpanel-testing                 
DEBUG … cabana2                                    
DEBUG stopping MariaDB server ...                  
INFO Done with migration to MySQL at /Users/robert/.LocalBeach/MySQL 
INFO If all works as expected, remove /Users/robert/.LocalBeach/MariaDB 
INFO Reading file from internal assets: local-beach/docker-compose.yml

However, directly after the run, mariadb is still running:

docker ps |grep database  
2611c71f28a8   mariadb:10.11 "docker-entrypoint.s…"   2 minutes ago   Up 2 minutes (healthy)   0.0.0.0:3307->3306/tcp, [::]:
3307->3306/tcp

kdambekalns · 2025-04-02T18:38:15Z

Hm… you did not do beach down before the setup, I guess? So that might be the "old one"? What happens if you do beach restart?

robertlemke · 2025-04-02T19:36:36Z

beach restart doesn't help, but beach down && beach start does. We should mention that in the messages after the migration.

robertlemke · 2025-04-02T19:37:21Z

kdambekalns · 2025-04-02T19:46:33Z

I didn't change anything:

Both work fine:

🤔

kdambekalns · 2025-04-02T19:49:06Z

Ah, SSL is disabled, I have it set to preferred… ✅

kdambekalns · 2025-04-02T19:50:34Z

beach restart doesn't help, but beach down && beach start does. We should mention that in the messages after the migration.

Or maybe even do a down before and tell people to run start afterwards?

robertlemke · 2025-04-03T12:27:10Z

Or maybe even do a down before and tell people to run start afterwards?

Wouldn't it be safe to run "down" and "start" (or just down) after the setup migrated the databases?

robertlemke · 2025-04-03T12:28:40Z

Ah, SSL is disabled, I have it set to preferred… ✅

Thanks, it works now with SSL set to "preferred".

The success message was always output, even when no migration was done.

kdambekalns · 2025-10-14T12:22:18Z

I just wondered, how this behaves if people do not run setup… it will probably lead to all databases seem to be gone.

Maybe we can add a check to the brew formula to output a hint when updating to a version greater than "last version using MariaDB"?

Ah, wait – does brew install run setup anyway?

kdambekalns · 2025-10-14T12:34:52Z

The setup runs automagically… when the "base" does not exist. So when upgrading, it will not run and people will have panic attacks due to "missing" databases (since MySQL will not have any data copied over).

- Add migration marker file to prevent re-running - Improve error handling with defer cleanup - Add progress indicators for each step - Add verification before suggesting data deletion

- Change bringBeachDown() to return error instead of bool - Consolidate compose file writing into writeComposeFile() - Update all callers to properly handle errors

Remove all DEFAULT clauses from JSON/TEXT/BLOB columns instead of trying to convert them, as MySQL 8.0 doesn't support default values for these column types in the same way as MariaDB. Handles both quoted strings and function calls like json_object().

robertlemke · 2025-10-14T14:30:17Z

Now, with these additions, Local Beach successfully migrated all of my databases and showed some progress while doing it.

kdambekalns

Nice with the progress and verification!

But of course I have something to complain about… 🙈

kdambekalns · 2025-10-14T15:01:32Z

cmd/beach/cmd/setup.go

-	if err == nil {
-		log.Info("Migrating MariaDB data from " + path.MariaDBDatabase + " to MySQL at " + path.MySQLDatabase)
-		log.Warn("Note: This may take a while, depending on DB size!")
+	migrationMarkerPath := filepath.Join(path.Base, ".mariadb-to-mysql-migration-complete")


Maybe put that into the MariaDB folder, so it is gone when the source has been removed? Or move it into the block that is run when there is MariaDB data to (potentially) migrate…

Otherwise the setup will always say the migration was completed, until the marker is removed.

Yes, good idea!

kdambekalns · 2025-10-14T15:02:29Z

cmd/beach/cmd/setup.go

+	if err != nil {
+		// No MariaDB data to migrate
+		if os.IsNotExist(err) {
+			log.Debug("No MariaDB data found, skipping migration")


Well, yes. But that will be shown for every setup run ever after… and even for someone who never even had MariaDB in her Local Beach setup.

Okay, let's remove that, it doesn't make sense.

kdambekalns · 2025-10-14T15:08:08Z

cmd/beach/cmd/setup.go


+	// Start Local Beach after setup/migration
+	log.Info("Starting Local Beach...")
+	err = startLocalBeach()


Does that work when it's not run in a Local Beach instance? I could imagine that when people run brew upgrade they might be in e.g. their $HOME…

It did work for me when I ran the beach setup in the Local Beach / Go project directory. So, there was no Local Beach project within reach.

kdambekalns · 2025-10-14T15:09:39Z

cmd/beach/cmd/setup.go

 			log.Info(fmt.Sprintf("  [%d/%d] Migrating database: %s", i+1, len(databaseList), database))
 			commandArgs = []string{"exec", "local_beach_database", "bash", "-c"}
-			commandArgs = append(commandArgs, "mysqldump -h local_beach_mariadb -u root -ppassword --add-drop-trigger --compress --comments --dump-date --hex-blob --quote-names --routines --triggers --no-autocommit --no-tablespaces --skip-lock-tables --single-transaction --quick --databases "+database+" | sed -e \"s/DEFAULT '{}' COMMENT '(DC2Type:json)'/DEFAULT (JSON_OBJECT()) COMMENT '(DC2Type:json)'/\" | mysql -h local_beach_database -u root -ppassword")
+			// Remove DEFAULT clauses from JSON/TEXT/BLOB columns for MariaDB→MySQL 8.0 compatibility


Not sure about this… it makes the migration work, but:

using sed on SQL might be doing very weird things

it will not fix the code that created incompatible SQL

Question is how we can fail gracefully then, with a meaningful error message? And you started the sed thing, didn't you?

kdambekalns · 2025-10-14T15:10:42Z

Let's let that sink in and also try to find a solution for the "setup isn't even run" scenario. No need to hurry now…

kdambekalns requested review from kitsunet and robertlemke October 11, 2024 15:24

kdambekalns self-assigned this Oct 11, 2024

Switch to MySQL 8

da4db23

With this Local Beach uses MySQL 8 instead of MariaDB 10 to be closer to the setup used in Beach. Running `beach setup` will migrate all databases from MariaDB to MySQL if the path used previously exists.

kdambekalns mentioned this pull request Apr 2, 2025

Update MariaDB from 10.11 to 11.4 (LTS) #93

Closed

kdambekalns force-pushed the feature/switch-to-mysql-8 branch from 2660e85 to da4db23 Compare April 2, 2025 15:10

Use mysql:8.0 to avoid getting 8.4

0a74a23

kdambekalns marked this pull request as ready for review April 2, 2025 15:36

kdambekalns added 6 commits June 24, 2025 15:25

Merge branch 'main' into feature/switch-to-mysql-8

101b3af

Merge branch 'main' into feature/switch-to-mysql-8

5cd7846

Reformat code

a24239d

Fix MariaDB migration success output

061749e

The success message was always output, even when no migration was done.

Move "beach down" code into helper function

2567a49

Bring Local Beach down before database migration

6e9bb8b

robertlemke added 2 commits October 14, 2025 15:59

Improve database migration safety and UX

8a4a8eb

- Add migration marker file to prevent re-running - Improve error handling with defer cleanup - Add progress indicators for each step - Add verification before suggesting data deletion

Refactor helper functions for better error handling

a53e8b8

- Change bringBeachDown() to return error instead of bool - Consolidate compose file writing into writeComposeFile() - Update all callers to properly handle errors

robertlemke added 2 commits October 14, 2025 16:26

Automatically start Local Beach after DB migration

e677c49

kdambekalns commented Oct 14, 2025

View reviewed changes

Uh oh!

Switch to MySQL 8 #103

Are you sure you want to change the base?

Switch to MySQL 8 #103

Uh oh!

Conversation

kdambekalns commented Oct 11, 2024

Uh oh!

kdambekalns commented Apr 2, 2025

Uh oh!

kdambekalns commented Apr 2, 2025

Uh oh!

robertlemke commented Apr 2, 2025

Uh oh!

robertlemke commented Apr 2, 2025

Uh oh!

kdambekalns commented Apr 2, 2025

Uh oh!

robertlemke commented Apr 2, 2025

Uh oh!

robertlemke commented Apr 2, 2025

Uh oh!

kdambekalns commented Apr 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kdambekalns commented Apr 2, 2025

Uh oh!

kdambekalns commented Apr 2, 2025

Uh oh!

robertlemke commented Apr 3, 2025

Uh oh!

robertlemke commented Apr 3, 2025

Uh oh!

kdambekalns commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kdambekalns commented Oct 14, 2025

Uh oh!

robertlemke commented Oct 14, 2025

Uh oh!

kdambekalns left a comment

Choose a reason for hiding this comment

Uh oh!

kdambekalns Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

robertlemke Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

kdambekalns Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

robertlemke Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

kdambekalns Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

robertlemke Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kdambekalns Oct 14, 2025

Choose a reason for hiding this comment

Uh oh!

robertlemke Oct 15, 2025

Choose a reason for hiding this comment

Uh oh!

kdambekalns commented Oct 14, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kdambekalns commented Apr 2, 2025 •

edited

Loading

kdambekalns commented Oct 14, 2025 •

edited

Loading

robertlemke Oct 15, 2025 •

edited

Loading