Migrating from1.12.2 to 1.18.1 issues

KevinTheJedi · Oct 3, 2024

I see you use LDAP. So you must first download the LDAP plugin for 1.18.x, place that in the plugins folder, and then import the db and upgrade. The updated plugin is needed during upgrade so it can convert the accounts accordingly.
Do you have any attachment storage plugins installed?
Read #1
Sounds like your Agent account doesn't have Department access to the rest of the Tickets. Add yourself to all Departments and see if you see all Tickets.

Also, I wouldn't update string from the config file. I'd simply copy over the entire config file and update just the db info if different.

Cheers.

glienhard · Oct 3, 2024

KevinTheJedi

Thanks for the reply. I'll answer each one here.

I did download the plugin for 1.18.x but didn't have it in the folder when I did the db import and upgrade. I'll wipe everything out and try it again with that plugin in the folder.
I do not use an attachment plugin. It is set to save attachments in the db with a 2Mb limit.
See #1
You hit that one on the head, it was totally a department access setting. In changing the setting I did notice that there is an inconsistancy with the departments. The local user that I'm logging in with had the primary department set to and active but private (internal) department. There was a red error on the page saying the primary department needs to be active. I did confirm it is set to active but it still does not show up in the dropdown list for primary department. It does however show for the extended access.

I thought the same thing about the config file but looking at it using diff there were so many other things in the new version I was afraid because I'm so far behind that something wouldn't work right if I kept the old file.

KevinTheJedi · Oct 3, 2024

glienhard

Try clearing your browser cache/cookies/sessions and hard reload the browser. If that doesn't work then I'm not really sure what it could be. You can login to your db, go to the _file table, look for type = 'L' for logs and type = 'B' for backdrops, grab the related id, go to the _file_chunk table, look for where file_id equals the ID(s) you copied, and see if it has data in the data column. If so then I'm not really sure what else it could be other than maybe corrupt export or something.
Idk it's starting to sound like something was upgraded correctly or exported correctly or something as if you see it as Active and Public under Admin Panel > Agents > Departments but still see the red warning for it everywhere else then something isn't right.

Are you certain you didn't have any customizations or anything to the previous codebase/database that might be hindering this upgrade/migration process?

Cheers.

glienhard · Oct 3, 2024

KevinTheJedi

I even tried loading up in an incognito window just to make sure I wasn't hitting a cache kinda issue. ¯\(ツ)/¯
I'll take a look in the db after my next attempt and see what I can find.

I do have a couple tweaks I have done to a couple php files but they are really basic things like changing wording of text.

glienhard · Oct 3, 2024

KevinTheJedi

I dropped the db and recreated it. Then imported the old db. Everything looks good so far.

Went to run the upgrade script and it is asking me to login (just same as last time). When I enter the local user credentials, it sits and spins. A page refresh just returns to the login. I re-enter the credentials and it spins again. I go to the address bar and remove the login.php so it is just tld/scp and then it loads the upgrade page.

Good news it appears ldap is working for staff, but bad news it's not working for users. Inspecting the db tables, it looks like the update script updated the backend field for _staff table but did not update it for the _user table. The _staff table backend field updated to ldap.p1i2 but the _user table stayed ldap.client so when I manually update a user to ldap.client.p1i2 that user can login. Interesting the user side logs right in with no issues.

I checked the _file table and found both the B and L files. Both files have data in the _file_chunk table. From the web interface I changed back to the stock logo and backdrop. I then deleted the custom entries. Verified the stock images are loading correctly. Uploaded a new logo and verified it saved to both tables. The logo shows as a broken image in the web interface. Changed logo for both Client and Staff to the custom logo and saved changes. Then tried loading the site in an incognito window, logo is broken.

KevinTheJedi · Oct 3, 2024

glienhard

Yea that logo thing is weird. Check all logs (general server logs, webserver error logs, PHP error logs, MySQL/MariaDB error logs, osTicket System Logs, Browser Console logs, etc.) for any related errors.

Cheers.

glienhard · Oct 3, 2024

KevinTheJedi

Okey dokey, looks like the image issue was self inflicted. I reverted all my php changes and now it's working correctly. I'll have to go through my code and see what/where it breaking.

Any thoughts on why the _user_account table didn't update the backend correctly with the update?

KevinTheJedi · Oct 3, 2024

glienhard

That’s exactly why I asked if you had any mods to the code/database.

Most likely a glitch during that upgrade. It can happen.

Cheers.

glienhard · Oct 8, 2024

KevinTheJedi

I did get everything working. Turned out I had a typo in how I added a comment to a php file and that was breaking everything

Still have the issue of the _user_account table not having the backend column updated by the update script.

Found one more thing. When viewing a ticket on the agent side that was brought over with the db import the 'Issue Description' field seems to be stipping off the " " tags that are saved in the db.

Here is what the value is in the db

Hi Dave, Please replicate the following tables from dataLink. Server: srv90037 Database: lccIRDataLinkDataWarehouse WAREHOUSE_FINAID.FA_AWARD WAREHOUSE_FINAID.FA_AWARD_SNAPSHOT WAREHOUSE_FINAID.FA_STUDENT WAREHOUSE_FINAID.FA_STUDENT_AWARD WAREHOUSE_FINAID.FA_STUDENT_AWARD_SNAPSHOT WAREHOUSE_FINAID.FA_STUDENT_AWD_CANCELED WAREHOUSE_FINAID.FA_STUDENT_NEED I just created a target file using the lccFileRowsTurnOnOff tool. The tables that need to be updated were all found in the tool's checkboxes. Thanks!

And here is how it's being displayed in the browser

Hi Dave, Please replicate the following tables from dataLink.Server: srv90037Database: lccIRDataLinkDataWarehouseWAREHOUSE_FINAID.FA_AWARDWAREHOUSE_FINAID.FA_AWARD_SNAPSHOTWAREHOUSE_FINAID.FA_STUDENTWAREHOUSE_FINAID.FA_STUDENT_AWARDWAREHOUSE_FINAID.FA_STUDENT_AWARD_SNAPSHOTWAREHOUSE_FINAID.FA_STUDENT_AWD_CANCELEDWAREHOUSE_FINAID.FA_STUDENT_NEEDI just created a target file using the lccFileRowsTurnOnOff tool. The tables that need to be updated were all found in the tool's checkboxes.Thanks!

Interestingly when I click to edit the ticket (as an agent), then everything displays correctly.

KevinTheJedi · Oct 9, 2024

glienhard

Share a screenshot of what you mean.

Cheers.

glienhard · Oct 9, 2024

KevinTheJedi

Sure no problem.

Here is a screenshot from the 1.12.2 system

Here is a screenshot from the 1.18.1 system (/scp/tickets.php?id=24766)

Here is a screenshot from the edit ticket page on the 1.18.1 system (/scp/tickets.php?id=24766&a=edit)

KevinTheJedi · Oct 9, 2024

glienhard

I figured that’s what you were talking about but wanted to make sure. That is the expected functionality. The display with HTML was causing issues so we started displaying it in plain-text only. If you want to see the full rendered HTML content simply click on it and it’ll bring up the preview window.

The only problem would be if the Agent didn’t have edit permissions for the Ticket or if the Field is not editable to Agents.

Cheers.

glienhard · Oct 9, 2024

KevinTheJedi

That is actually a huge problem for us. Seems like turning off/on the display of HTML should be up to us. Do you have any recommendations for how to fix this? I cannot go live with the upgrade until that is fixed (it's that big of an issue).

KevinTheJedi · Oct 9, 2024

glienhard

You’d have to customize the code for that. There is no option for this; won’t be an option until after v2.0.

Cheers.

glienhard · Oct 9, 2024

KevinTheJedi

OK, thank you. Couple more thoughts and questions about this.

When did this get changed? From your stand point, would there be any issues with grabbing the code from the 1.12.1 system and using it in the 1.18.1? I mean just the part for displaying this with HTML.

I'm also a bit baffled as to why we have a wysiwyg type box on the user side for them to type in all kinds of fancy HTML kinda stuff and then completely strip it out when the agent views the ticket. Just trying to understand the bigger picture as to why this change was made.

BTW, huge thank you for all your time in the forum here and all your work on this!

KevinTheJedi · Oct 9, 2024

glienhard

A while ago; you’d have to go look at the commit history for exact date.

No, as there is more to it than that. Textarea fields have their own AJAX endpoint and all that. So a bit will need to be reworked.

You can still see the full rendered content you just need to have edit permissions so you can simply click the content and get a full preview window.

The issue was we truncate textarea field content in the header (as it could get way too long) and if you truncate HTML it’s incomplete and breaks the whole page as it too is HTML. We also have inline edit now so you don’t edit the full ticket anymore for simple field updates. Each field value is clickable so you can update just that field. So with that being the new norm it made sense to display partial plain-text first then you click it to see the full rendered content.

Cheers.

glienhard · Oct 9, 2024

KevinTheJedi

Thank you for all that help. Given I'm on a dev server right now I was able to test out and idea that seems to be working and wanted to let you know what I did.

In the file include/staff/ticket-view.inc.php I found where the Laravel function Format::striptags() is being used to strip out all the HTML. Since this function does not have any built-in control options I switched over to using the PHP funtion strip_tags(). Here is the snippet of what I did that is working for me.

foreach ($displayed as $a) {
    $id =  $a->getLocal('id');
    $label = $a->getLocal('label');
    $field = $a->getField();
    $config = $field->getConfiguration();
    $allowedTags = '<p>,<br>';
    $html = isset($config['html']) ? $config['html'] : false;
/*  $v = $html ? Format::striptags($a->display()) : $a->display(); */
    $v = $html ? strip_tags($a->display(), $allowedTags) : $a->display();
    $class = (Format::striptags($v)) ? '' : 'class="faded"';
    $clean = (strip_tags($v, $allowedTags))
        ? ($html ? strip_tags($v, $allowedTags) : $v)
        : '&mdash;' . __('Empty') .  '&mdash;';
/*  $clean = (Format::striptags($v))
        ? ($html ? Format::striptags($v) : $v)
        : '&mdash;' . __('Empty') .  '&mdash;'; */

KevinTheJedi · Oct 9, 2024

glienhard

That’s not a Laravel function; it’s a custom osTicket function We had to define everything ourselves back then; we didn’t use any frameworks.

Cheers.

glienhard · Oct 9, 2024

KevinTheJedi

I did find where that is being defined. I removed my previous edits made changes appropriate to where the static function is built.

Here is what I have done. I added a variable to the include/staff/ticket-view.inc.php for $allowedTags and then added this variable to the places where Format::striptags is called. I then added the strip_tags function inside the custom static function. This would allow the flexibility of defining the $allowedTags per item, per page. Snippets are below

include/staff/ticket-view.inc.php

$allowedTags = '<p>,<br>';
$v = $html ? Format::striptags($a->display(), $allowedTags) : $a->display();
$clean = (Format::striptags($v, $allowedTags))
    ? ($html ? Format::striptags($v, $allowedTags) : $v)
    : '&mdash;' . __('Empty') .  '&mdash;';

include/class.format.php

static function striptags($var, $excludedTags=null, $decode=true) {

    if(is_array($var))
        return array_map(array('Format','striptags'), $var, array_fill(0, count($var), $decode));

    return strip_tags($decode?Format::htmldecode($var):$var, $excludedTags);
}