My Project Lithium Battery Caught Fire

Yep...

 

I can send you my Signal Soother.

 

A member has already offered to send me a Signal Soother for evaluation.
I will post a video with my findings once it arrives in the mail.

 

Can you please give an update on the Signal Soother evaluation?

 

I posted my Signal Soother Analysis video yesterday in Post#67 in this thread. Grab your popcorn and a hard hat if you choose to read the lively followup commentary.

 

OP: If you'd rather, I can post this video in a new thread. I believe it makes sense to post here, since it is a failure analysis based on the hardware that was probably installed in your vehicle when your thermal event occurred.

...

Here's my NexCell V2.8 hardware review:

tl;dw: The NexCell V2.8 electronics have the same fundamental architecture as the V1 unit I reviewed previously. Specifically, there is no supervisory control feedback mechanism, hence the NexCell modules are unable to alert the vehicle when certain failure conditions occur.

The primary difference between V1 & V2.8 is the cell balance voltage thresh hold. In short, whereas V1 only balanced cells when they were essentially overcharged, V2.8 now continuously discharges down to approximately 70% SoC (I explain my 'approximately' caveat in the video). Honestly this probably introduces more issues than it solves, particularly if the "connector inside the module came unplugged" failure mode occurs. Again, this is discussed further in the video.

Before replying with non-scientific ad-hominem attacks, please heed the ground rules I laid out in the first couple minutes of the video. For those that don't want to give me those sweet sweet likes, basically know that I won't respond to ad-hominem attacks in any meaningful way. If you have a scientific counterpoint to anything I claim in the video, I 100% welcome that feedback. Otherwise, please know that you are wasting your keystrokes.

...

Since @AzusaPrius almost certainly won't heed my ground rules, I'm going to preemptively carry over the previous counter I started in post #126 in the Signal Soother thread:
Qualified scientific statements @AzusaPrius has made: 0
Qualified scientific statements @AzusaPrius has addressed: 0

 

All that matters here is energy storage needs to be engineered You can't just start putting little bags of lithium in plastic housings etc . It's a whole package oh well there's always that Why can't somebody come up with a packaged battery similar to the way I don't know all of this nonsense is being made now out of these around the cells that look like fat AA batteries or whatever they are. Why are we dealing with little sacks of jail stuffed and plastic injection molded housing etc I don't know I don't really care I just need a battery that works without a lot of hoopla seems to me the Toyota battery is just that now why somebody else can't come along and make a battery in a similar shape with the newer type of cells that folks are generally using now and be done with it $211 volt-based blah blah blah what is so complicated about this obviously something

 

I have a 2008 Chevy Tahoe hybrid which uses 40 of the same modules as a Prius. In Oct 2022 I purchased a set of batteries for around $5000 Canadian as I was unable to get an OEM NiMH pack and had been burned buying reconditioned modules. At the time, I wasn't able to find any negative reviews. I put barely around 10K miles on the Tahoe before the lithium batteries cratered. I went to look into warranty, then went down the rabbit hole and discovered all the recent failures of his battery packs and I found that he is recalling batteries (which he didn't bother to email or call to inform me of). I messaged Jack asking for a refund instead of a replacement pack of batteries as it appears that the V2.5 doesn't fix the issues with the batteries I already have and was told that he is broke and unable to issue refunds. He offered to ship a V2.5 pack more or less immediately or to wait until he announces a V3.0 pack supposedly within the next month and that he will ship one of those to me instead.

I'm not so sure I want to put a V2.5 into my vehicle as there is no BMS to limit cold weather discharge (it is below -10F here for 8 months of the year and drops to -40F at the coldest) and now I've found a report of a fire. My vehicle is now parked outside and is non-operational until I either get a new battery pack or get my hands on a good OEM pack to put into it instead. I'm not so sure I want to put a V3.0 into my vehicle until a 3rd party has reviewed it and deems it to be properly designed and implemented.

With Jack claiming to be broke, and me being an international customer, I don't expect I'll ever get a replacement battery or a refund. That $5000 could have been far better spent on the things my family of 5 needs rather than on a junk product that doesn't do what it is claimed to do. I feel so incredibly ripped off with no recourse.

 

IMO using second-sourced lithium modules from reputable battery manufacturers is the correct method for aftermarket lithium conversions. You need economies of scale that simply aren't available unless you're a billion dollar auto manufacturer.

I'm excited to eventually offer this product to the 3rd-party market. Specifically, I will sell a "DIY LiBSU Kit", which includes everything required to perform the conversion, except for the lithium modules themselves. I will design the LiBSU Kit to work with several generally available lithium modules salvaged from newer electric/hybrid vehicles.

For those that don't want to perform the conversion themselves, I am partnering with a well known aftermarket battery company, who will sell a "Drop-In Lithium Battery" for NiMH Toyota vehicles.

I intend to initially support for the Gen3 Prius, followed by the Gen2 Prius. No concrete plans after that, but there's no reason I couldn't offer similar variants for any NiMH Toyota. This will of course take time.

Based on my experience exploring the parallel G1 Honda Insight market over the past couple years, designing into this space is an interesting market position. I suspect the scar NexCell leaves on the aftermarket community is going to take some effort to overcome.

Honestly I've already done 90% of the leg work with my existing LiBCM design (for the Honda Insight). In fact, I'll be able to use the LiBCM circuit board as a functional prototype in the Gen3 Prius... of course I'll need to spin the circuit board to make it fit nicely, but no sense doing that until I've properly decoded the serial data (shouldn't be that difficult based on what I've seen so far).

 

Sorry to hear your $5000 pack ended up costing $0.50/mile to operate.

Rather than answer your other statements line-by-line, I've opted to just generally reply to your post with more information based on what I've heard about NexCell circa 2024JUN:

Jack is really in a corner right now. From what I've heard, he's absolutely overwhelmed by his existing product's ~30% RMA rate. Unfortunately, at this moment he doesn't have a 'safe' design (i.e. one with supervisory control when a lithium cell is 'unhappy'), so the only thing he can do right now is ship out the same flawed hardware. He previously had enough headroom to offer customers (who asked directly) for a refund, but my understanding (and your confirmation) is that that hasn't been an option recently.

Jack is also simultaneously attempting to salvage the NexCell name with a new V3 design, which he is promising will (finally) have the safety elements required in a commercial lithium product. I would hope that he has enlisted a 3rd party engineering company to create said product. My understanding is that this V3 redesign is well underway, and could be announced in the next couple months. However, given that Jack and I were still discussing me creating said 'safe' V3 product as late as 2023DEC, I would be skeptical with that timeline, unless Jack started from an existing 3rd party design (e.g. my open source LiBCM product, or an existing design from another 3rd party consultant).

If the V3 hardware becomes available, rest assured I will provide another technical analysis.

IMO Jack's one-on-one private damage control strategy is an even worse business decision than failing to properly design the product in the first place. Jack has fabricated many tall tales, sometimes in public view, but more often during private conversations. I'm still debating whether or not I want to explore the technical merits behind these statements (in a separate thread).

Based on the options given:
-$0 refund, or;
-V2.5 pack now, or;
-V3 pack later, or;
-OEM pack not available.

Ideally you'd just get a full refund (alas, not an option). Based on NexCell's financial uncertainty, I certainly don't recommend waiting for V3. This leave the only the "V2.5 pack now" option, which is debatably better than nothing. At the very least, you'll be able to drive the car again until the pack fails. And if it does fail during your remaining warranty period, then maybe Jack will send you a V3 replacement then (if it becomes available). So as I see it, "V2.5 pack now" is your best bet.

You don't. I know this contradicts my previous paragraph, but it sounds like your best option right now. Maybe another alternative is to get the replacement V2.5 blades and then sell them to NexCell fanboys before the market craters. Obviously I would recommend advising anyone you're selling to to look up the safety issues, but if you can find a buyer that wants to throw caution to the wind, at least you could recover some of your investment. Maybe V3 will be compatible with the existing V2.5 mechanicals (although any V3 design that didn't have at least one more stud to mount the OEM BMS wire onto almost certainly wouldn't be safe).

If you aren't driving your vehicle (i.e. it's bricked because the battery is dead), then parking outside is probably an overly-cautious. The LFP chemistry is substantially safer when it's sitting unused for an extended period. The primary safety issues I've identified are most likely to occur while driving, particularly during heavy regen; or immediately following reavy-regen driving events.

Based on the V1 & V2.8 PCBs I've reviewed, I propose that every PCB Jack has released will safely bring any overcharged cells down into the safe operating area after just a few hours parked. Therefore, it is probably safe to park your vehicle indoors after three hours, and is certainly safe after parking overnight. Driving the car a few hundred feet (e.g. from outside the garage to inside the garage) isn't going to appreciably decrease your safety margin.

THe only caveat I'll add to this statement is that Jack has acknowledged a manufacturing issue where the 'BMS' connector that connects each cell to the PCB might not be plugged in on some units. If that is the case, then an overcharged cell would stay overcharged, which does pose a safety hazard. So then ideally you'd want to visually verify every single module's 'BMS' connector is plugged in, which would require you to remove four screws from every blade and then verify said connection.

However, that's impractical, so a faster (but less thorough) test can be performed by simply reading each blade's voltage. If all the blades are at similar voltages, that would suggest all the connectors are plugged in. This test doesn't guarantee all cells are in a safe operating area, but it substantially lowers the uncertainty, to a point where I would be comfortable parking a 'bricked' Toyota in my garage.

In summary, a thermal event is most likely to occur while driving, primarily due to undetected cell failure during heavy regen events.

It goes without saying that if you still have the Signal Soother installed, you should banish that thing from existence ASAP.

I will review V3 if it gets released and someone will send me one. If @jacktheripper wants to provide me a pre-release unit, I promise I'll give an impartial review on the (hopefully) new circuitry. I doubt that'll happen, so then I'll need to wait until someone accepts my $250 bounty for a V3 unit (send me a PM if you're this person).

Jack's "I'm broke" claim is probably the most honest, straightforward statement I've heard come out of his mouth.
You got swindled and I hope you are made whole. I'm not optimistic, but if Jack is able to pull out of the nosedive I know he wants to right his product's shortcomings... I'm just not sure he has the runway to do so at this point; a 30% RMA rate is a dire situation.

 

Sorry, I haven't read past page 2, it was just going into incrimination rather than the actual thing that happened.

Firstly, that cloud is not smoke, it is boiled electrolyte, the result of over voltage charging on an individual or cells, not the whole pack.

How does it happen, running the voltage to each cell well beyond the stable upper limits, go over 3.6v in any cell and you are inviting voltage run away that in turn becomes thermal run away.

I did point out the risk of this happening some time back when I was building my LTO battery pack. You can't have regen in the high amps and expect a balancer that moves milli amps to balance the cell top end. A cell going high voltage should trigger the regen to stop.... the problem is, the whole regen module/battery voltage is not a fixed number, it allows for over voltage on regen because the old module chemistry and design enabled them to dissipate some over voltage as heat and boiling the electrolyte a little ... you just can't do that with LFP cells, boiling the electrolyte, even a bit, looks just like the vapour cloud that started this thread.

If those cells were rebalanced, that battery would actually function again, just with a lot more resistance across some cells and lost capacity in those cells ... weed them out and replace them with good cells, and the battery would work again ... much like swapping out failed modules in the old batteries .....

T1 Terry

 

Hi Mudder, I posted asking for help on your LiBCM tech support thread on InsightCentral day before yesterday because software update for Linsight didn't work. I also private messaged you yesterday. Can you please help me get your product working again? It's really frustrating that you're ignoring me after my friend spent so much money on your product. My friend had to rent a car for her roadtrip because your Arduino firmware expired and I can't get the updated firmware to load? Can you please help me? I've been trying to get it to work going into day 3 now and I just need a simple reply to tell me what to do to get the firmware to load?

I've asked everyone on Insight Central for help and I'm not getting anywhere with them or your customer service for your product and it's really frustrating!

I thought maybe you were just traveling or something and couldn't get to it yet. But now I get on PriusChat and it looks like you're more interested in attacking Jack's work than helping customers with your own work?

What's up with that? Please! I really need your help! Your updated firmware won't load and the car won't work without it. Please help?

 

Ok, I can at least substantiate there is such an issue raised by a "Prius Camper" over at insightcentral.net on a thread called "LiBCM Open Beta Support Thread":

LiBCM Open Beta Support Thread | Page 66 | Honda Insight Forum

From what I can tell from the argh.txt file that was attached (which isn't much, being all unfamiliar with the stuff), there seem to be errors in compiling, rather than in getting the compiled firmware to load.

 

Yes... His beta firmware has an expiration date and I went over to my friend's house who loves her Insight and loves his Linsight product when there was 9 hours remaining and after some hassles with the USB cable, the firmware install failed and I posted for support before I went to sleep on Thursday night asking for help.

Here we are on Saturday morning and still no reply for why Arduino firmware update won't install and my friend has to rent a car for her road trip because Arduino won't install and rather than a reply to help me out I come here and see how much time he's put into bashing Jack's work. And that's fine, he's entitled to his opinions, but when my friend who has his product in her car (a very excellent product by the way) can't get support because he's too busy complaining about his future competitor's product rather than supporting his own product. It's really frustrating!

I love the quality of work and design that Mudder does for Honda Insight. It's high quality, but Arduino are for hobby use and are not high quality and not for commercial use and I don't want to argue or get mad or anything, I just want to get my friend's Linsight firmware updated because it's expired and has shut down the system and she can no longer use her beloved Insight and had to rent a gas guzzler for a really long drive.

UPDATE: He finally replied to my support request and I documented everything and replied to his reply, but still need another reply to figure it out. Sure hope I don't have to wait as long for the next reply?

UPDATE2: He replied to my reply and there's a bunch of things I can try and will report back later once I have time to try them.

 

I see over there that mudder has also concluded the issue is a compilation error, prior to even attempting to load anything onto the board. Apparently whatever got downloaded and unzipped won't compile, and I hope that'll be easy to sort out.

 

I've tried three times already... His YouTube tutorials are usually very helpful and was able to install the firmware successfully a couple months ago after a bunch of difficulty on my side.

I think part of the problem might be the video tutorial is outdated. He's telling me I may of not unzipped the files properly, but I've done ever step in his tutorial several times with the same result. I think there must be a change in the files I'm supposed to select since he made his tutorial video?

I need to take a break and relax and stop being upset about letting my friend down by failing to update the Firmware. Once I calm down enough I'll try the tutorial for a 4th time and maybe I'll get lucky? This is my first experience with Arduino and I think I'll forever avoid it if I can. It may be great for hobby products, but when I've asked my friends who are experts with Arduino for help they don't have much to offer other than wondering out loud why he'd be using a hobby product for a commercial product.

I mean his LIBCM board is gorgeous and his overall design work of his kit gets an A+, but his dependence on an Arduino hobby-based microprocessor rather than just putting his own processor on the board has been nothing but a massive hassle at every step for me and it's not the first time I've waited multiple days to get tech support for Arduino problems with him.

 

Bring popcorn if you do decide to keep reading.
The "actual thing that <probably> happened" is pretty well summarized in this reddit post.

Correct, and it smells terrible. I was able to quickly reproduce this failure mode on my front porch in about fifteen minutes.

Correct. In my testing I had to get to a slightly higher voltage, but that is immaterial to the discussion, given that NexCell cannot directly monitor each cell, and also lacks a supervisory control mechanism to tell the car something is wrong.

I (obviously) agree. It's one thing to DIY a one-off pack that you then use knowing these limitations... but it's something else entirely to spin up an empire with such a shoddy product. Eventually the wheels were going to fall off... right now I think we're down to the last wobbling lug nut.

This is probably the most concise "why is NexCell a bad idea" I've read so far. It 100% addressing the ultimate failure mode. Kudos for summarizing it so well.

Correct. It's refreshing to write that.

I think I agree with what you're saying... to clarify, you're saying "you could reuse all the modules except for the one one that vented, right? I wouldn't recommend reusing a vented cell (which is probably quite swollen). Regardless, this is moot for OP, as they dropped the battery off at a government hazardous goods location.

 

In the future, please consider keeping your Honda Insight related posts on that forum. I don't see any appreciable reason to cross-post here, except to sow doubt amongst the Prius community regarding my ability to support my LiBCM customers. Rest assured I have a stellar support record and reputation in the Honda community; I think with one or two insatiable exceptions, you will find that my support capabilities are well received over on ic.net.

Per the terms of my LiBCM Beta Agreement (which you agreed to), you know that I do not provide free private technical support; all posts must be made in the correct "LiBCM Open Beta Support" threads. Note that this clarification is irrelevant, as I did not check ic.net yesterday, due to my travel plans and then vacation. In other words, yesterday I received neither your correctly posted public support request nor your private message.

I have responded to your multiple requests. Please continue the discussion in the correct LiBCM Support thread on ic.net.

I'm not ignoring you. You need to adjust your support expectations. I am allowed to take a day or two off here and there. You are being unreasonable with your numerous simultaneous requests for the same support issue. Specifically, this is too much:
public post to LiBCM Open Beta Support Thread: 2024JUN14@04:26
public post to LiBCM Open Beta Support Thread: 2024JUN14@18:09
private message to me directly on ic.net: 2024JUN14@18:15
private message to me directly on ic.net: 2024JUN14@18:16
public post to LiBCM Open Beta Support Thread: 2024JUN14@19:56
public post to LiBCM Open Beta Support Thread: 2024JUN14@20:59
public post to LiBCM Open Beta Support Thread: 2024JUN14@21:57
private message to me directly on ic.net: 2024JUN15@14:13
public post to LiBCM Open Beta Support Thread: 2024JUN15@14:30
public post to an unrelated PriusChat thread: 2024JUN15@15:05
private message to me directly on ic.net: 2024JUN14@15:11
public post to an unrelated PriusChat thread: 2024JUN15@16:42
public post to an unrelated PriusChat thread: 2024JUN15@17:29

...making all these posts for the same issue is unreasonable.

Your friend did not have to rent a car; the insight functions just fine with expired firmware. Regardless, I don't understand why your friend didn't contact me instead of you. Given that you also own an LiBCM unit, I'm curious how it is that you have the firmware running in your car, but unable to get it working in your friend's car? Who is your friend?

Note that the reason the firmware is expiring is because LiBCM is still in Open Beta. During this period, the firmware expires approximately every 40 days for safety reasons. Specifically, I want to make sure that customers are always running the latest firmware version. I knew that if I didn't force customers to upgrade (i.e. by disabling the firmware), then many customers would never do so. Hence my decision to mandate reinstallation every 40 days. This annoyance will disappear once I am certain LiBCM's firmware is safe. Your friend agreed to these terms when they signed up for the LiBCM Open Beta.

Regarding "going into day 3 now", correct me if I'm wrong, but the first time you brought this to my attention was:
public post to LiBCM Open Beta Support Thread: 2024JUN14@04:26

If that's the correct first contact, then note that my first response was 'only' 35 hours later (at 2024JUN15@15:13). Normally I try to respond to all requests within 24 hours, but in this particular instance I was traveling during your initial contact. Regardless, while you may have struggled for three days (not ideal), I would not have known that fact until 35 hours prior to my response.

I understand your frustration, but in the future please consider sending at most one "please help" message per 24 hour period, unless you are responding to my followup message. Also, if your friend is going on a trip and needs to get it done sooner, please let me know initially, so that I can prioritize my support (if I see it in time).

I want to reiterate that LiBCM is a Beta product.

I have thoroughly answered this assertion over on ic.net... the first place I read it. If you want to continue this particular discussion, please do so here, as it relates to Prius content (hence it has no place over on ic.net).

...

Finally, I want to stress that I pride myself in the support I offer to my customers. In your case I can see how you are frustrated, but I hope you understand why I think your support expectation timeline was unreasonable.

 

Note that I'm only repurposing the Arduino hardware. Both the bootloader firmware AND the actual LiBCM firmware are essentially running on bare metal C (rather than the Arduino 'Wires' programming language). There are a couple trivial exceptions to the previous sentence, but they don't matter from a safety perspective.

Also, so far my Arduino hardware failure rate is QTY1 (i.e. one customer). This was an EEPROM memory corruption issue, which the LiBCM hardware detected and reported to the customer (and also disabled the IMA system for safety reasons). I then had that customer send me the board for failure analysis, and was able to confirm that this was simply a one-off latent failure. In other words, I've had exactly one Arduino-specific hardware failure after ~1.14 million operating hours.

I did.

Given what I've written today, do you think this question is in good faith?

 

Please continue this support request in the correct LiBCM Open Beta Support thread. It has no relevance here. If you continue to believe I am ignoring you on ic.net - and somehow feel this is pertinent to the Prius community - please feel free to post here again.