November 2023 Server Migration Notes

Discussion in 'PriusChat Website Questions' started by Danny, Nov 30, 2023.

  1. Danny

    Danny Admin/Founder
    Staff Member

    Joined:
    Nov 24, 2003
    7,094
    2,116
    1,174
    Location:
    Charlotte, NC
    Vehicle:
    2013 Prius Plug-in
    Model:
    Plug-in Base
    Alright, FINALLY! After almost 3 months (and several years of starts and stops), PC has been moved to it's new home on a more spacious server in both SSD space (960 GB NVME) and memory (64 GB). The 2 Mac Minis that drove PriusChat for many years did their job mostly well and will now be retired.

    There is a much larger update coming to the forum software that PriusChat runs. We are currently on a 1.x.x version of the software and will be updating to a 2.2.x version of it in the coming days/weeks as I get more testing out of the way.

    Emails should be working again.

    Please post any issues that you come across in this thread.

    I'll try to sit down and get to some of your concerns and questions that were raised in the other thread in the near future, but here's the long and short of it:

    I started this site in my early 20s. I'm now in my mid-40s. Pretty much everything in my life has happened since I started PriusChat. I got married, then we had a daughter, then we had another daughter, then we had a third daughter. My oldest is now 13 and youngest 7. They are finally of the age where I feel like I have more time for my own projects again (but I will certainly miss those younger years).

    As my life's expenses have increased, PriusChat's revenue has decreased. I decided to shut down the store because I found that many people have become awful when they think they are just talking to an email account. It was no longer worth the cost of my mental health. Ad revenue is not what it once was, as has been documented all over the internet. And so I had to start looking for new revenue streams, which has led me to my current business venture somewhat adjacent to my background in IT. It's been great, but it has also taken up almost all of my time with extensive travel and large amounts of stress.

    I have received numerous offers for PriusChat over the years, and have always declined because I didn't like the idea of a "corporation" controlling what should be a community.

    PriusChat will never be sold. If it is ever a project I no longer have interest in, I will gladly give it to the community to keep alive. To many it may just be a set of files and a database, but to me it is my life's work. I don't have a Prius right now, and that keeps me disconnected from the community, so I am sorry for not being an active member among you. I have a bit of a mental block that if something isn't perfect with the site, I can't be on it unless I'm working on fixing every little thing. I'll try to be a more active participant, and appreciate you all for keeping the site alive and moving forward.

    This migration took so long because I wanted it to be perfect; in hindsight, I should have remembered that PriusChat is never going to be perfect, which is what will always make it home.
     
  2. ChapmanF

    ChapmanF Senior Member

    Joined:
    Mar 30, 2008
    25,076
    16,345
    0
    Location:
    Indiana, USA
    Vehicle:
    2010 Prius
    Model:
    IV
    I have a bad feeling from looking at this post here, where the image (that showed up before migration) is now a broken-image icon.

    The URL is to an attachment from another post, in the form https://priuschat.com/proxy.php?image=https%3A%2F%2Fattachments.priuschat.com%2Fattachment-files%2F2021%2F10%2F213026_3A8F3E73-169F-46CA-8FC5-BE8A962C183F.jpeg&hash=2d1e25c772ff24ea472482b7f286cb5c

    The failure may mean that the image files didn't all get migrated into the right places for proxy.php to find them, or the hashes don't match what they were before, for some reason. Or there's a database table of images and their hashes and its contents didn't get migrated.

    It might be something pretty straightforward to fix, or more of a headache; it may affect a whole lotta posts with links to images.

    Edit: yup, it's something to do with the cache; if I ditch the proxy.php and the hash and just ask for the original image URL (https://attachments.priuschat.com/attachment-files/2021/10/213026_3A8F3E73-169F-46CA-8FC5-BE8A962C183F.jpeg), that image is still there.

    I am guessing that if I mention that image here:

    [​IMG]

    it will get cached, and should have the same hash it had before, which might then make the original post I linked to work again, and everyone will see no problem with it and think I was imagining things.

    Edit again: welp, that didn't happen. Instead there's also a broken-image icon here. The plain URL that I entered has been replaced with a proxy.php URL with a hash (same hash as in the original post), but the image still isn't retrievable from the cache for some reason ... right after the migration it wasn't, and if my posting here made an attempt to re-cache it, that didn't work either.

    This particular case (a proxy.php cached copy of an image that's hosted on PriusChat to begin with) isn't even my biggest worry if the cached images didn't get migrated across. My bigger worry is all the cached copies of images people linked to from elsewhere on the web, some of which might no longer be available at their original URLs.

    That goes especially for all posts where people linked to parts.toyota diagrams, as none of those exist at the original source anymore.
     
    #2 ChapmanF, Nov 30, 2023
    Last edited: Nov 30, 2023
  3. Danny

    Danny Admin/Founder
    Staff Member

    Joined:
    Nov 24, 2003
    7,094
    2,116
    1,174
    Location:
    Charlotte, NC
    Vehicle:
    2013 Prius Plug-in
    Model:
    Plug-in Base
  4. ChapmanF

    ChapmanF Senior Member

    Joined:
    Mar 30, 2008
    25,076
    16,345
    0
    Location:
    Indiana, USA
    Vehicle:
    2010 Prius
    Model:
    IV
    As I edited into post #2, I'm even more worried about images that were in the cache and are no longer accessible at their original URLs. Those can't get rebuilt over time. That would be a big loss. All links anybody ever made to parts.toyota diagrams are probably in that category.
     
    gggGary likes this.
  5. PriusCamper

    PriusCamper Senior Member

    Joined:
    Mar 3, 2012
    11,447
    4,649
    0
    Location:
    Pacific Northwest, USA
    Vehicle:
    2007 Prius
    Model:
    Two
    Thanks for all your hard work! And thanks for giving us a much needed perspective about what the past 20 years of running PriusChat has been like for you! This community has been truly blessed by you!
     
  6. Danny

    Danny Admin/Founder
    Staff Member

    Joined:
    Nov 24, 2003
    7,094
    2,116
    1,174
    Location:
    Charlotte, NC
    Vehicle:
    2013 Prius Plug-in
    Model:
    Plug-in Base
    Looking at the logs, it looks like the issue with your photo not showing up is an outlier. I have all of the data backed up in case there are issues we run into. The proxy is set to rebuild the cache every 7 days, so let's see if it rebuilds your photo on Tuesday.
     
    gggGary likes this.
  7. sylvaing

    sylvaing Senior Member

    Joined:
    Jul 15, 2023
    1,187
    496
    0
    Location:
    Canada
    Vehicle:
    2017 Prius Prime
    Model:
    Plug-in Base
    Thanks Danny. I'm quite a new user here as I have my Prius Prime only since July but the wealth of knowledge and community involvement here is awesome. I'm glad you're keeping it alive and don't plan on letting it die if you decide to stop. Thanks again!
     
    chinarut, gggGary and Function like this.
  8. Tideland Prius

    Tideland Prius Moderator of the North
    Staff Member

    Joined:
    Oct 2, 2004
    45,046
    16,263
    41
    Location:
    Canada
    Vehicle:
    Other Non-Hybrid
    Model:
    N/A
    Photos aren't uploading @Danny . I tried both pasting the photo directly and saving it/uploading it via "Upload a file"
     
    gggGary, Mendel Leisk, hill and 2 others like this.
  9. Kiwi7910

    Kiwi7910 Junior Member

    Joined:
    Oct 25, 2023
    49
    17
    0
    Location:
    New Zealand
    Vehicle:
    2007 Prius
    Model:
    N/A
    I am no stranger to burnout. I retired early from a 20 year career in software development and currently work part time, providing IT support for my local high school. Computer science is discipline that requires 100% focus and maintaining that focus can be extremely stressful.
    If I were you, I would be looking for ways to offload what you can to suitably qualified volunteers within the Prius Chat membership. People management has it's own problems, but effective delegation is the key to success when a project gets to big for one person alone to handle.
    Thanks for all the hard work so far, you are not alone...
     
    chinarut and Danny like this.
  10. Danny

    Danny Admin/Founder
    Staff Member

    Joined:
    Nov 24, 2003
    7,094
    2,116
    1,174
    Location:
    Charlotte, NC
    Vehicle:
    2013 Prius Plug-in
    Model:
    Plug-in Base
    I'll look into this today, thanks @Tideland Prius .
     
    fuzzy1 and Tideland Prius like this.
  11. ChapmanF

    ChapmanF Senior Member

    Joined:
    Mar 30, 2008
    25,076
    16,345
    0
    Location:
    Indiana, USA
    Vehicle:
    2010 Prius
    Model:
    IV
    'Outliers' in database migrations usually concern me. Until some reason has been shown why a migration treated https://attachments.priuschat.com/attachment-files/2021/10/213026_3A8F3E73-169F-46CA-8FC5-BE8A962C183F.jpeg differently than umpty thousand jpeg files before it and after it, I tend to be kind of on edge wondering how many other outliers are going to be discovered in the days/weeks/months to come.
     
    #11 ChapmanF, Dec 1, 2023
    Last edited: Dec 1, 2023
  12. bisco

    bisco cookie crumbler

    Joined:
    May 11, 2005
    110,443
    50,202
    0
    Location:
    boston
    Vehicle:
    2012 Prius Plug-in
    Model:
    Plug-in Base
    That was well said Danny. Thank you for the many years of hard work keeping the show going .
    When you have time, can you explain why paid subscriptions never happened?
    I could be wrong, but I think most users would be happy to sign up
     
    Danny and PriusCamper like this.
  13. nerfer

    nerfer A young senior member

    Joined:
    Mar 1, 2006
    2,507
    237
    28
    Location:
    Chicagoland, IL, USA, Earth
    Vehicle:
    Other Hybrid
    Model:
    N/A
    I know I appreciate keeping PriusChat away from the corporations that already control too much. I was on another car site that got hacked, turned out they had a dozen or more car forums and all the user information was in one location. When that was compromised the whole chain was compromised. Also, what would a large owner with corporate policies do with something like FHOP?

    And I kinda like the idea that I met Danny in person (many, many years ago), can't say that about a corporate site.
     
    chinarut, gggGary, Austi and 2 others like this.
  14. tochatihu

    tochatihu Senior Member

    Joined:
    Apr 10, 2004
    9,218
    3,593
    0
    Location:
    Kunming Yunnan China
    Vehicle:
    2001 Prius
    @ChapmanF , would disappearing images be at archive.org? Or some other similar place? Not something I can investigate because archive.org is blocked in some countries. Yeah that thing again.

    ==
    Anyway our @Danny deserves more praise than will probably accumulate in this thread. If a subscription option is developed, I'm in.

    ==
    This is probably not the place to talk about developing a book to publish about Prius and Priuschat. But it is an idea that has occurred to me. I doubt I am alone in that.
     
    hill likes this.
  15. ChapmanF

    ChapmanF Senior Member

    Joined:
    Mar 30, 2008
    25,076
    16,345
    0
    Location:
    Indiana, USA
    Vehicle:
    2010 Prius
    Model:
    IV
    When I place a bare URL into a post, like this one:

    https://priuschat.com/wiki/blink-a-k-a-flash-codes-how-to/

    before the migration, it would show up in the post as the title of the wiki page. Now it just shows up as the bare URL.

    As far as I'm concerned, either way gets the job done, but I thought it might be worth mentioning the changed behavior.
     
    Danny likes this.
  16. ChapmanF

    ChapmanF Senior Member

    Joined:
    Mar 30, 2008
    25,076
    16,345
    0
    Location:
    Indiana, USA
    Vehicle:
    2010 Prius
    Model:
    IV
    That would depend on whether (or how recently) archive.org had spidered the pages where those images appeared. And I don't think anyone is going to want to discover one-by-one which PriusChat posts are now showing broken images, and somehow determine whether they were just broken pre-migration or not, and then try to retrieve those from archive.org and put them back where they belong on the server (or even try to automate that).

    There ought to be a much easier way assuming @Danny has a complete backup of the former server.

    I'm still a bit spooked by the suggestion that the one image I noticed first may be an 'outlier'. My own history with such migrations has left me thinking that noticing one 'outlier' or mishandled file, after what was to be a systematic migration, is like noticing one cockroach, and might merit some investigation into just how it could have happened and how much else could have been affected.

    I don't mean any of that to sound critical. I know what kind of a job it is. Thanks @Danny! I'm just trying to help identify loose ends, ideally before the new server has diverged so far from the migrated state that solutions get harder.
     
  17. rjparker

    rjparker Tu Humilde Sirviente

    Joined:
    Jun 6, 2008
    8,701
    5,190
    7
    Location:
    Texas Hill Country
    Vehicle:
    2012 Prius v wagon
    Model:
    Three
    Having trouble adding pics.


    Error message:

    The following error occurred
    There was a problem uploading your file. Img_3645.jpeg
     
    gggGary, Chimi1 and Onlooker like this.
  18. Mendel Leisk

    Mendel Leisk Witness Leader

    Joined:
    Oct 17, 2010
    57,104
    39,425
    80
    Location:
    Greater Vancouver, British Columbia, Canada
    Vehicle:
    2010 Prius
    Model:
    Touring
    ^ Ditto. It's not just a Texas thing.
     
    Onlooker likes this.
  19. Mendel Leisk

    Mendel Leisk Witness Leader

    Joined:
    Oct 17, 2010
    57,104
    39,425
    80
    Location:
    Greater Vancouver, British Columbia, Canada
    Vehicle:
    2010 Prius
    Model:
    Touring
    Thursday at 9:16 pm:
    Friday at 3:40 am
    Might be a while.
     
  20. sleekitwan

    sleekitwan Junior Member

    Joined:
    Feb 4, 2023
    24
    12
    0
    Location:
    UK
    Vehicle:
    2010 Prius
    Model:
    Business Edition
    I’d like to thank Danny for creating the forum, and it is the quality of the senior participants that helps make it a reliable and the goto resource for Prius issues and interesting discussions. The migration of any IT system, is a little fraught, I have done a couple, but small-scale stuff and it’s stressful, somewhere, at some point, every time.

    Now a bug report. I did do an item and wanted to upload an image, explaining the exact mistake I made that caused a certain fault, this being me mis-placing two 30-Amp fuses in the Prius gen 3 fusebox. I was told to use the ‘upload file’ option when on my iPad, and then I get the option to ‘choose photo/take photo,etc’. I choose a photo and just get an error message with a red band horizontally saying cannot upload or something, but it does give the exact filename of the image in question.

    That’s it, images do not upload from my iPad. They did before. Thank you and everyone be safe and keep well as the winter sets in. My best for the festive season, to both sides of the Atlantic, and around the world.
     
    Danny likes this.