The last 24 hours have been the biggest challenge that Onell has faced since our first day on the web.
What started out as a great release, quickly transformed into a near mind breaking disaster. The depth of the duplicate Paypal order issues that occurred last night completely crushed our long trusted internal operation.
Every single system we have been employing through Paypal for years imploded all at once, with even our inventory count becoming paralyzed, resulting in massive discrepancies with our available stock and what was actually sold.
After trying to reconcile the seemingly endless problems that have been caused by the meltdown, we decided after speaking with Paypal on three separate calls, that a master refund had to be activated.
An extreme series of events had to be met with an extreme response, and we made this painful decision for the benefit of all of us. Michelle had even gone as far as to invoice over 350 orders, after I had already issued over 160 refunds, and discovered that nothing was lining up with our stock, due to the automatic Paypal inventory freezing just after the store went live.
I so deeply apologize for what occurred with this release. We are working with the Paypal techs to figure out why their system imploded on us this particular time, when all other signs had been pointing to more efficient and easy launches.
We are hoping to "relaunch" later this coming week, but only if we are 100% satisfied that this level of pain cannot occur again.
I'm sorry for the mess, but we will clean it up. I promise each of you that.
Please consider sticking with us while we navigate this negative zone. Your support keeps us from literally going mad with bloody knuckle frustration over what transpired last night.
More details the minute we get them.
EDIT: ALL ORDERS PLACED SINCE LAST NIGHT HAVE BEEN COMPLETELY REFUNDED.
UPDATE: We are still working with Paypal to discover what caused the problems last Friday night, but have nothing solid to report at this time. All the feedback from the release that has been sent in by you guys has also been turned over to the Paypal techs, so we really appreciate the information.
Once again, I truly apologize for the depth of what's occurred. Your support in the comments below and through personal emails have kept all of us over here from going into some very dark places.
6.15.2013
Subscribe to:
Post Comments (Atom)
Posts
Are you canceling all orders placed and starting fresh or just canceling the duplicates ?
ReplyDeleteHey Chris. Every single order since last night had to be refunded. We could not get the numbers to line up no matter what we tried.
DeleteThat's sad and understandable. My only worry is that the drop will be rescheduled before the Paypal Reversal/ refund is credited back to our accounts
DeleteThe refunds should go through pretty quick, if not instantly. I had already issued over 160 refunds and just about all of them instantly were credited back. Though this whole experience has shaken our long relationship with Paypal in general.
DeleteThere are so many risks never seen by the public eye in regards to operating a small company. Just a few well placed strikes can compromise everything, bringing even a reliable operation to its knees.
The level of embarrassment that we feel over here is almost unbearable, especially after working so hard to develop Onell into something we trust to deliver the goods.
I will be apologizing for this debacle for a long time to come.
All of my refunds (4 duplicate orders, yeesh) came through instantly. Being on the boards pretty much through this whole thing, I have not run across one person that feels you have any reason to be embarrassed, or that this reflects negatively on Onell at all.
DeleteEveryone knows that it was paypal's mess, but and that you got stuck cleaning it up.
This whole time I (and I know I'm not the only one) have mostly felt sorry that you guys have had to spend your whole weekend working this out, and deal with the ensuing headaches.
Same here. This hasn't shaken any trust or good will, though here's hoping that, if the system needs to be changed in any way, it is not nearly so painful a process as the last 48 hours. You all do such hard work over there, and I sincerely hope everything works out for the best despite what's happened. Good luck in sorting it all out, and know that you'll still have some of the most committed fans (and friends) in the business.
DeleteMatt, while we are sad as a result of the issues, everyone knows that this is not a problem/ fault of you guys. The problem instead rest squarely on the shoulders of PayPal, and how a big company whose purpose is sales/ payments could fail.
DeleteYou need not apologize
Thanks for the heads up, Matt. It's too bad this all happened, but we all have your back. I'll still be making my purchase, as I'm sure many others are. Keep us updated and keep on trucking!
ReplyDeleteI'll second this. Hope it's up sooner than later for everyone involved.
DeleteI got a refund notification for my actual order, so I assume the same is of everyone.
ReplyDeleteAw, that's a shame. Hope it all works out okay, I'm pretty psyched to get a pair of Nobotos. :) Hang in there, team!
ReplyDeleteBummer, but I think a lot of us had a feeling that this was probably the only way to fix the problem. That stinks that the inventory monitor froze. Makes me worried how quickly noboto and the new armorvor really sold out.
ReplyDeleteKeep your heads up, hopefully paypal makes it right and have treated you guys fairly. When the new drop is announced I'll do all I can to be there for it.
Hope that you guys are hanging in there...can't imagine the level of torment that must've been. Talk about a bad day at work!
ReplyDeleteThat little Noboto...ripped a hole in the fabric of time and space in what is now now as the Paypalous Rift!
Hope things get back to normal for you guys and you get a chance to regain some sanity.
I"m not going anywhere... though I'm sad I'm going to need to try and secure at Noboto again, ditto the armorvor... but it's always worth it
ReplyDeleteI hope it's soon!
http://www.youtube.com/watch?v=uNe7x13q2MU
ReplyDeleteWe live to play another day :)
Glad to see this all got fixed, no worries at all, my money will be waiting till then....
I feel worse for you guys than I do for myself. I'm patient enough to wait for my toys, but this had to be a total nightmare for your guys. Thank you once again for rectifying the problem as swiftly and efficiently as possible, and I wish you all luck in getting things up and running again.
ReplyDeleteHey guys, don't sweat it. We're with you for the long haul.
ReplyDeleteSad to see that you guys had such a frustrating issue pop up with this round of releases. After hearing that other people were having issues I was glad to see that my payment had gone through ok and was admittidly a little bummed when I saw the refund. Although it had to understandably be extreme, I'm glad you figured out a fair solution to the issue and hope things go smoother the next time around. I'm sure I speak for all of us when I say we value and appreciate all the hard work you and everyone has put into your company and the always beyond upstanding way you guys run it. Silly glitches are easy to forget when so many great previous experiences drown them out. keep up the great work!
ReplyDeleteHeh. Freakin' Noboto.
ReplyDeleteSkip the embarrassment and stop the torment -- we're still here for you guys.
Hell, this just means I get a drop night adrenaline rush much sooner than expected.
Crap... I just hope I manage to get a noboto or two on the re-release. ;_;
ReplyDeleteSooooo sorry to hear this bro! I think everyone is 100% behind you guys and will return to support the drop.
ReplyDeleteI'm beginning to think that very few internet payment systems can handle a sudden spike in focused orders at a single time. Sure Amazon is rock solid, but they don't do pre-announced drops. I suspect if they did a "drop" the way those of us do for a really hot item that even they could have issues. Real time online inventory control when you have a strictly limited amount of product is very difficult. About the only system I can think of that regularly does this stuff in real time mostly successfully is Ticketmaster...
No problem. I understand.
ReplyDeleteTake care Team Onell! We'll be here!
ReplyDeleteJust a bump on the road.
ReplyDeleteDon't feel embarraced. We're all friends here.
Matt,you guys kick butt! You guys are of the greatest examples of customer service Eva! We know you got this:)
ReplyDeleteBuy from you guys again?
ReplyDeleteReady when you are!
Bring on the goodies!
;-)
I'm with you guys too. Can't wait for new toys. Keep on rocking!
ReplyDeleteThis is unfortunate but I'm really just sorry that this has caused so much trouble for the Onell crew. No one blames you guys at all.
ReplyDeleteIMO, Onell is the gold standard of great drops and fan relationships. This is just a small bump in what has otherwise been an amazing journey.
You're the best at what you do, don't let the failings of others (::: cough paypal cough:::) get you down. We will be ready and waiting for the next deployment.
Michelle, Matt,
ReplyDeleteI couldn't say better than Cappy. You can rely on our unwavering support, Onell team rocks. 'nuff said :)
I echo the others when I say don't worry about your reputation. I know it feels like a big deal to you but from our perspective it's pretty minor. We'll just reorder whenever you're ready!
ReplyDeleteIt's even less stressful knowing that the quantities on most of these are higher than normal (minus Noboto and Armovor I assume). I'm sure we'll all be able to get our Pheydens.
Now that the secret of Noboto is revealed, I bet there will be more people around waiting for the re-drop than there were at the original drop on Friday night. I'll be hugely disappointed if I'm unable to snag a Noboto and an Armorvor at the re-drop, but I certainly wouldn't hold it against Onell, and it definitely wouldn't hurt my enthusiasm for future drops, either.
ReplyDeleteWith all the excitement over Noboto, I wouldn't be upset if Matt changed the limit to 1.
DeleteThat would be quite reasonable.
DeleteSounds good to me too...
DeleteYou guys are awesome, we know it. PayPal just can't keep up with your awesomeness, so don't sweat it. We'll all still be here when they get it figured out. :)
ReplyDeleteI must echo everyone else here. The Onell crew is amazing, and this bout of trouble only cements the fact that they strive to be fair, forthcoming, and responsible to the fans!
ReplyDeleteThank you so much, and I hope you all dont kick yourself too much about it, we all see that the fault isn't yours! We are behind you all the way!
Matt in addition to all that everyone has said already I just want to add Happy Father's Day. I hope you were able to spend time with the girls instead of this issues. We will all be here ready and waiting when the redrop happens. You can count on us.
ReplyDeleteI sincerely hope that all goes well as you untangle the mess PayPal has caused. And despite the OD charge I just took due to PayPal's error, I eagerly await the just.
ReplyDeleteMuch love Matt. This is just a hiccup.
ReplyDeleteNo worries, Poop happens.
ReplyDeleteHope everything gets fixed and ready for a 2nd go soon.
Good luck guys!
This comment has been removed by the author.
ReplyDeleteNot going anywhere, I know you guys work your asses off! And it was paypals fault. Only thing I'm bummed about is trying to free up the time on a friday night for another drop. I usually work then. But, like with every Glyos drop, its always worth it cause the goods are awesome and you guys are a awesome company!
ReplyDeleteMan PayPal is starting to turn into a evil thang. I just found out that my order was cancelled, I can imagine how the weekend went for ya'll. I hope you at least got some fun on Fathers Day Matt !!! No need to be sorry guys we'll all be here when the drop comes...
ReplyDeleteHiccups/poop/stuff/things happen. No worries!
ReplyDeleteMatt, dun worry bout it! The wait just makes me want to throw money at yo faces all the more. Hopefully no more hiccups happen. You guys are great!
ReplyDeleteHey, I just got an email from Matt's address that was just some gibberish, a very sketchy looking link and his name. Is something up with your yahoo account?
ReplyDeleteHe got hacked.
ReplyDelete