[3.0] Miscellaneous fixes for the installer and upgrader#9162
[3.0] Miscellaneous fixes for the installer and upgrader#9162Sesquipedalian wants to merge 23 commits intoSimpleMachines:release-3.0from
Conversation
|
So... Test here next??? |
|
On this PR... Results are actually very good. Only a couple minor nits & questions. 2.1 => 3.0:
Install: Ran fine. Only minor nits...
General observations... These are both maybe just me & my system, not sure... Any thoughts would be welcome.
I think the performance is actually a 3.0 thing. I don't see that elsewhere. I think the self-signed cert is a me thing... I see that elsewhere. No matter what I've tried, if I try to force https, I get sslRedirect within the app. And the installer never detects the cert, e.g., the ssl box is grayed out. I hate putzing with certs... |
|
I'd be OK with this going in as-is. Only nits. |
|
I could dive deeper & try 2.0 upgrades if needed. Sound like a plan? |
Hm. That's not good.
Should be an easy fix.
I haven't noticed that in my tests, but I wasn't particularly looking for it either.
Yeah, that's probably just a you thing. Rather than a self-signed certificate, why not just use one from Let's Encrypt? |
Sounds like a great plan! Thanks, @sbulen. 😊 |
Most of my testing is on localhost. Once things progress, I do final testing on a hosted test environment. For a long time, no issues whatsoever, but browsers have been getting more picky over time. For years, a self-signed cert worked perfectly. Until it didn't... For a while there, I had to create my own CA for local use. Now, that is either unsupported, or, I'm doing it wrong... |
2.0 latin1 => 3.0:
php/apache/mysql logs: Nothing upgrader log:
2.0 utf8 => 3.0:
php/apache/mysql logs: Nothing upgrader log:
pg => 3.0:
php/apache/mysql logs: Nothing upgrader log:
MariaDB 11 => 3.0:
php/apache/mysql logs: Nothing upgrader log:
|
Sesquipedalian
force-pushed
the
3.0/upgrader
branch
from
1e36b28 to
727233c
Compare
These two should be fixed in the latest commits. |
Hm. Does the admin_info_files table not exist in the database before the upgrade starts? |
Bleh. Please run it again with this modified version of ./Sources/Maintenance/Migration/v2_1/CustomFieldsPart1.php. It will once again dump a bunch of crap into upgrade.log so I can figure out where things are going wrong. <?php
/**
* Simple Machines Forum (SMF)
*
* @package SMF
* @author Simple Machines https://www.simplemachines.org
* @copyright 2026 Simple Machines and individual contributors
* @license https://www.simplemachines.org/about/smf/license.php BSD
*
* @version 3.0 Alpha 4
*/
declare(strict_types=1);
namespace SMF\Maintenance\Migration\v2_1;
use SMF\Db\DatabaseApi as Db;
use SMF\Db\Schema;
use SMF\Maintenance\Maintenance;
use SMF\Maintenance\Migration\MigrationBase;
class CustomFieldsPart1 extends MigrationBase
{
/*******************
* Public properties
*******************/
/**
*
*/
public string $name = 'Upgrade Custom Fields (Preparing)';
/****************
* Public methods
****************/
/**
*
*/
public function execute(): bool
{
Maintenance::$tool->logProgress(__LINE__, true);
$start = Maintenance::getCurrentStart();
if ($start <= 0) {
$table = new Schema\v2_1\CustomFields();
Maintenance::$tool->logProgress(__LINE__, true);
$table->normalize();
Maintenance::$tool->logProgress(__LINE__, true);
$this->handleTimeout(++$start);
}
if ($start <= 1) {
Maintenance::$tool->logProgress(__LINE__, true);
$table->populate();
Maintenance::$tool->logProgress(__LINE__, true);
$this->handleTimeout(++$start);
}
if ($start <= 2) {
Maintenance::$tool->logProgress(__LINE__, true);
// Add an order value to each existing cust profile field.
$ocf = $this->query(
'SELECT id_field
FROM {db_prefix}custom_fields
WHERE field_order = 0',
);
// We start counting from 5 because we already have the first 5 fields.
$fields_count = 5;
Maintenance::$tool->logProgress(__LINE__, true);
while ($row = Db::$db->fetch_assoc($ocf)) {
++$fields_count;
Maintenance::$tool->logProgress($row['id_field'], true);
$this->query(
'UPDATE {db_prefix}custom_fields
SET field_order = {int:field_count}
WHERE id_field = {int:id_field}',
[
'field_count' => $fields_count,
'id_field' => $row['id_field'],
],
);
}
Db::$db->free_result($ocf);
Maintenance::$tool->logProgress(__LINE__, true);
$this->handleTimeout(++$start);
}
return true;
}
} |
Sesquipedalian
force-pushed
the
3.0/upgrader
branch
2 times, most recently
from
2ab8fe3 to
e9dbf3c
Compare
Yes, the table exists, & the prefix is specified in Settings.php. It's a simple 2.1 install. I suspect the problem here, though, is that somehow the prefix was lost... It's not "admin_info_files", it's "smf_admin_info_files". |
|
Ah, yes. That would likely explain it. I'll take a look when I have time later. |
MariaDB 2.1 => 3.0:Refreshed the code, implemented your tweak above & reattempted the upgrade. It actually got much further... 
php/apache/mysql logs: Nothing upgrader log (bottom):
|
Should be fixed in latest commits.
Should be fixed in latest commits. If not, things might get a lot harder. Ready for another round of testing, @sbulen. 🙂 |
3.0 Install:
2.1 => 3.0:
2.0 latin1 => 3.0:
php/apache/mysql logs: Nothing upgrade.log (bottom):
2.0 utf8 => 3.0:
php/apache/mysql logs: Nothing upgrade.log (bottom):
pg 2.1 => 3.0:
php/apache/mysql logs: Nothing upgrade.log: MariaDB11 2.1 => 3.0:
php/apache/mysql logs: Nothing upgrade.log (bottom):
|
|
Thanks, @sbulen. It's been a while since I have been able to look at this PR again. I appreciate both your help and your patience.
Should be fixed in latest commit.
For diagnostic purposes, please run it again using this modified version of ./Sources/Maintenance/Migration/v2_1/CreateAlerts/php:
For diagnostic purposes, please run it again using this modified version of ./Sources/Maintenance/Migration/v2_1/PostgreSqlSequences.php:
For diagnostic purposes, please run it again using this modified version of ./Sources/Maintenance/Migration/v2_1/CustomFieldsPart1.php: |
|
Suggestion: Maybe we should just keep the debug stuff in there. That will help reduce back & forth. Consider even making it a switch. Can't have too much diagnostic info & options in these upgrades, IMO... 2.0 latin1 => 3.0:
php/apache/mysql logs: Nothing upgrade.log:
2.0 utf8 => 3.0:
php/apache/mysql logs: Nothing upgrade.log:
pg 2.1 => 3.0:
php/apache/mysql logs: Nothing upgrade.log (portion - this went on for days...):
MariaDB11 2.1 => 3.0:Doesn't look like if failed in the same place... Not sure why: Note the weird display - all the debug lines are still on the same line, no line feeds: 
php/apache/mysql logs: Nothing upgrade.log (bottom only): |
|
A clue, not sure how much it helps.... The generic 'Critical Error' errors appear to be on the calls to handleTimeout() within a step. More likely to happen when attempting a restart. Sometimes, it just seems to fall thru to exit... Trace follows... This logic is new to me, but it seems to be checking for json in $_GET, that isn't there? |
|
OK, did some more sleuthing, & have a couple of things to share...
I'm not quite sure what the fix is unless the upgrader can force a hard refresh to ensure new .js files are loaded. I think this is causing all of the CRITICAL errors w/o messages.
...is being rejected in MariaDB. I'm not sure why exactly yet... Maybe a default or a SQL_MODE issue set for the session. Works OK without the quotes, but I see the same behavior in MySQL. Something appears to be making them behave differently at runtime. |
|
That is all very helpful, @sbulen. Thanks! I'll see what I can do about them. |
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Fixes error when upgrading from SMF 2.0 Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
|
Reran some of the above, ensuring current script.js before starting to run the upgrader, to get a better pic of where we're at. It doesn't get rid of all of the Critical Errors as I had hoped, but the picture is a bit clearer. I'll see if I can drill down on the utf8 issue. 2.0 latin1 => 3.0:
2.0 utf8 => 3.0:
2.1 pg => 3.0:
2.1 MariaDB11 => 3.0:
|
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
|
Here's what I think is going on with a few of these... The problem appears to be in checkAndHandleTimeout() in ToolsBase.php, when it thinks it's run out of time... THAT's when it passes thru & checks json. All of the prior calls to checkAndHandleTimeout() return OK, but they don't fall thru to the json check. If it falls thru to the json check, it just exits. So it's a timing thing, which is why it's inconsistent. |
|
I wonder if the best thing for the js issue is to always force it to load a fresh file and not from the cache. In theory, once the upgrader starts, we would only load that a few times before JS takes over to run the migrations and cleanups. |
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Sesquipedalian
force-pushed
the
3.0/upgrader
branch
from
b61882e to
280a584
Compare
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
Signed-off-by: Jon Stovell <jonstovell@gmail.com>
|
Upgrading PM labels, adding support for alerts, fixing PostgreSQL sequences, and the problems with cached JavaScript files should all be fixed in latest commits. The |
2.0 latin1 => 3.0 upgrade:2.0 utf8 => 3.0 upgrade:Both fail in the same location & with the same message. Note: I really like the error, very helpful... 
|
2.1 pg => 3.0:
|
2.1 MariaDB11 => 3.0:Still seeing the same error. I'll see if I can drill down on this further. 
If you just hit "Try Again" it proceeds further, but has a similar issue a little further along: 
|
|
Nit, but I see in my access logs that we are mixing http query delimiters: See also: #9140 |
|
The problem with the MariaDB NULL is that they are stored in information_schema differently. The queries used in the normalize function return different values. So... When comparing the structure with expected values, the upgrader thinks the database needs to be updated, and executes the commands to do so, and fails for MariaDB. These commands aren't executed for MySQL, because they match the expected values. I'm actually not sure the update would work for MySQL either, because it's not getting executed... That would need to be tested as well. In the screenshot below, the structure from a MySQL DB is shown via a print_r, and on the right the same structure from a MariaDB via a print_r - as returned from Table->getCurrentStructure(). Note that almost all the columns are subtly different, and in fact, these are detected as changes and commands to modify the columns are executed. Not just NULLs, but empty strings are stored a bit different (see the name column). Most appear to be working, the null defaults aren't. So, these weren't sql_mode subtleties at all, the sql_mode was consistent for both. It has more to do with how stuff is stored in information_schema. 
|
3 participants
Fixes a bunch of stuff that has come up while testing the installer and upgrader.
@sbulen, we can use this PR to do all our testing to get 2.1 → 3.0 and 2.0 -> 3.0 upgrades fully working, as well as deal with any issues in the installer. Then we'll go back to #9120 afterward.