Rendered at 22:13:11 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
dgl 2 days ago [-]
I tried using their Magic Containers product and there were issues that showed a lack of attention to detail as well.
It's supposed to scale globally (magically!) but I found multiple cases where particular nodes were problematic and the health checks didn't detect them (in fact to start with the health checks didn't even work properly if you had multiple containers, they did fix that). The support was quite slow too, after finding multiple product issues they'd escalate to developers and then come back a month later and ask to retest, but some of this took multiple round trips. I was only using this on a side project, but definitely wouldn't consider them for anything critical, even if they are quite cheap.
"does not matter if this is an isolated edge case. Data loss for 15 months with poor support isn't something that can be waved off as 'edge case'. The fact I had to go to Reddit to get someone's attention for this is insane. By the way, I tried reaching out to various 'senior management' personnel via LinkedIn last year and no-one replied. Escalation requests via the Support thread ignored and declined. Very professional of you guys."
and that seems pretty damn reasonable of a reply. the fact they had to go to reddit is insane. a sarcastic "very professional of you guys" is pretty tame, given the situation, and not discrediting at all.
subscribed 2 days ago [-]
You did a commendable job of ignoring all the op's (from Reddit) explanations and context to come to this weird output.
OP even actually responded to a similar comment in this reddit thread - look it up and tell me he wasn't reasonable. 15 months of fobbing him off, and once the case has been raised publicly the very senior guy _suddenly_ sees it as a priority despite it being just an edge case.
Really? Do you work for their PR or what?
manytimesaway 2 days ago [-]
> 15 months of fobbing him off, and once the case has been raised publicly the very senior guy _suddenly_ sees it as a priority despite it being just an edge case.
That's basically how this whole industry works? How would you justify the outrageous cost of product management layers otherwise?
Not saying that this is the right way. But it's not a very surprising story, no matter which company (be it Google, Microsoft, OVH...).
JCattheATM 2 days ago [-]
> That's basically how this whole industry works?
Far from it. Most companies in the same space don't just ignore such significant problems. If they did, they wouldn't survive for very long.
manytimesaway 24 hours ago [-]
I don't know, Google, Microsoft and Oracle seems pretty resilient then.
JCattheATM 22 hours ago [-]
Got an example of them acting the same way as Bunny in the case being discussed??
manytimesaway 16 hours ago [-]
Do I really need to pull out all of the exemples of Google support being the absolute worst on Earth?
Microsoft arbitrarily banning IPs/AS from its infrastructure with no recourse or acknowledgement of the issue (such as 365 lying about accepted mail traffic through SMTP)?
JCattheATM 9 hours ago [-]
> Do I really need to pull out all of the exemples of Google support being the absolute worst on Earth?
For paid products, denying or lying about there being any issue as serious as significant data loss for over a year like Bunny did, yes.
> Microsoft arbitrarily banning IPs/AS from its infrastructure with no recourse or acknowledgement of the issue (such as 365 lying about accepted mail traffic through SMTP)?
That sounds closer but still not comparable.
subscribed 10 hours ago [-]
FWIW I had Gmail lying exactly the same - back then I had a Googler actually check the logs and confirm.
I've got a 250 and the ID, then the server discarded the email.
Imustaskforhelp 2 days ago [-]
I have heard OVH support be tricky but personally, I have talked to one of the main devs in their discord server for something not even too tricky, but they were kind enough to just walk me through it.
I am not sure but I do think that they might have a ticket system there too.
Either way both Hetzner and OVH would respond in better technical rigour
I used to like bunny but this ain't a good look :-(
throwaway81523 2 days ago [-]
I've exchanged emails with Dejan (Bunny CEO) about Bunny-related stuff. It's not THAT hard to contact him, or at least wasn't, some years back. Maybe he's a bigger cheese now.
wat10000 2 days ago [-]
Pretty mild when getting a reply that puts more energy into downplaying the problem than acknowledging that they've had an ongoing data loss issue for over a year and their support has been useless.
Maybe the data disappeared because BunnyCDN towed it out of the environment.
bakugo 2 days ago [-]
It's a perfectly understandable response to what is essentially just corporate damage control. I'd be pissed too if I was in their position.
They've been ignoring the issue for over a year, yet it somehow took them only 4 hours after the reddit post was made to determine that it's "an isolated edge case", even though there's at least one other user reporting the same problem. The comment was clearly written with a focus on saving face rather than properly apologizing for the terrible support.
bobdvb 2 days ago [-]
Yeah.
Someone recommended Bunny to me and I looked at it previously based on that but didn't have a use case then. But as someone who regularly recommends vendors for v.big CDN contracts I'm not impressed with their attempts to downplay things.
RobRivera 2 days ago [-]
Handwaving of professionally agreed upon SLAs as an edgecase and providing poor customer support; thats a paddlin
m3nu 2 days ago [-]
Also a user of their CDN, storage and container service for about 4 years.
My experience was more positive. One time I had a minor issue with their storage where I couldn't replace a file or something. This was fairly early after the product launched. They fixed it and gave me free credit for reporting it.
DANmode 2 days ago [-]
If the story is true, remaining a user seems…terrifying?
ipaddr 2 days ago [-]
Because they fixed a minor issue many years ago?
zamadatix 2 days ago [-]
I believe "the story" was intended to refer to the horror story in the main post rather than the minor issue described in the comment above.
DANmode 2 days ago [-]
100%
reddalo 2 days ago [-]
That's scary. I'm in the process of moving all of my services to Europe, and I had considered BunnyCDN, but after this I'm not so sure anymore.
I also tried Hetzner Object Storage. I love Hetzner, they're great, except for their Object Storage service, which is completely unreliable (errors, slowness, etc.). I'm surprised that Hetzner still hasn't retired that product until it's properly fixed.
My last chance is with Scaleway. Xavier Niel's products are always good, so fingers crossed...
WhyNotHugo 1 days ago [-]
Scaleway is a disaster in terms of handling payments.
At some point there was a bug in their payment system and I couldn't pay an invoice. The payment failed the first time due to lack of funds on my account (this initial failure was my fault). Once my account had funds, retrying failed with a vague error on Scaleway's side.
I reached out to support, and it was a known limitation: if your initial payment fails, you an only retry with a subset of their supported payment methods. I didn't have access to any of these payment methods, and I made it clear that I could pay, but only with the payment methods we'd been using thus far.
They kept sending me emails reminding me of this overdue bill and threatening to send collection agency, all while I kept exchanging messages with support complaining that their system gave an internal error.
The bill was, by the way, for less than €10. But a collection agency will charge you hundreds for the "collection" service.
They were aware of the error in their payment system, but it seems they had no intention of fixing it. After months of back and forth, they just cancelled the invoice.
The amount of time and stress in dealing with was ridiculous.
miyuru 2 days ago [-]
>My last chance is with Scaleway.
I have been using for sw for object storage for a while. I haven't had a a issue with them with object storage.
only problem I had was with a another product called cockpit(they ship metrics about object storage there), which they bundled with object storage product, which cannot be disabled.
This month I had a 12 euro surcharge, because they enabled something in cockpit by default a while ago and suddenly start to charging for it.
champtar 2 days ago [-]
There is also OVH (not affiliated, just happily using their VPS), but I would consider switching to 2 providers, as any provider can have data loss or just lock your account at anytime for any reasons. 2 providers with free egress so you can easily replicate data between them.
WhyNotHugo 1 days ago [-]
The classic rule is 3-2-1:
- Three copies
- Two types of media
- One off-site or offline
12907835202 2 days ago [-]
I guess you may have alot of files, but to me object storage is so cheap I keep copies on Aws S3, wasabi, r2 and a 16 TB HDD on my hetzner server.
Admittedly 4 might be too many. But at some point I switched to r2 for the free public egress and deleting one of Aws or wasabi has never been a priority and I don't want to do it without putting in the time to quadruple check I'm not deleting anything important.
But at the very least id have 3. I'd hate to discover that my S3 was blown up in a war and then copying everything my HDD was the last straw that pushed an aging drive over the edge.
pier25 2 days ago [-]
Bunny's CDN is great. The issue in the post is about their file storage service not the CDN.
For my storage needs in the EU I'm using Upcloud's object storage. Very happy with them. Then a Bunny cdn zone if I need to share the files publicly.
s09dfhks 2 days ago [-]
Interesting timing given the post yesterday about someone switching to BunnyCDN from cloudflare
Havoc 2 days ago [-]
To be fair the Reddit user account is 8 years old. Not conclusive but does suggest it may be organic
sergiotapia 2 days ago [-]
you can purchase reddit accounts with age/content.
DANmode 2 days ago [-]
That is interesting.
httpsterio 2 days ago [-]
Not a great way to handle the issue originally, but I think the CEO's response is more than fair.
I haven't had to contact Bunny's support so it's a bit disappointing to see this type of runaround from the support, but as an actual issue it seems pretty minor in the end.
I've been using Bunny for ~four years after looking at video hosting. In the four years we've saved approximately 50 000€ compared to if we'd gone with Cloudflare video streaming service and it's just been rock solid for us.
mort96 2 days ago [-]
The CEO's response is alright, the VP of engineering's response is a bit more worrying in how it mainly tries to downplay the issue.
getcrunk 2 days ago [-]
I’ve used bunny for a few years … happily. I wonder if this is a bug due to some meta data of the files like the names or something. Very weird. Good thing you had metrics to catch it.
I upload all object storage stuff to bunny for live but also to backblace for backup.
I’ve always wanted to implement fail over client side for any asset over to bacblaze but seems like a lot of overhead
kjs3 2 days ago [-]
Wait...this has been going on for 15 months without resolution or recompense and you haven't pulled up stakes and moved to almost anyone else? I get "it's a lot of work to move" and maybe "we hope they'll figure this out so we don't have to move", but in my world that excuse runs out waaaaaay before 15 months. The people I'm accountable to would have hauled me out back and put me out of their misery after, maybe, 6 months on the outside.
WhyNotHugo 1 days ago [-]
I'd guess that it hasn't become an issue with any of their clients. If any big client had been impacted once, they would have sped up moving. If a big client had been impacted twice, they would have moved already.
kjs3 22 hours ago [-]
That's fair, and I agree (hope?) if it was customer impacting there would be a miraculous allocation of scarce resources to move platforms. But in my world I'd get asked "why are you still screwing around with a vendor that clearly sucks" even if it was for a non-prod playpen of no consequence. YMMV.
simoncion 2 days ago [-]
> ...but in my world that excuse runs out waaaaaay before 15 months.
I expect the combination of
Yes, we should've migrated away sooner, we never had the capacity to do so and hoped Bunny would just get their shit together.
and
The loss rate isn't enormous in percentage terms, but it's consistent and ongoing.
means that detecting and dealing with the loss is substantially less work than moving away. [0] I expect that Management is fully aware of what's up and is making the call here.
[0] "Just" add a retry if the post-upload verification step fails! Sure, it's slower, but it works, right??? mournful sob
I'm a very grateful bunnycdn customer. they are great, lovely UX and great performance and price. we use them to store our image files and other documents.
It's supposed to scale globally (magically!) but I found multiple cases where particular nodes were problematic and the health checks didn't detect them (in fact to start with the health checks didn't even work properly if you had multiple containers, they did fix that). The support was quite slow too, after finding multiple product issues they'd escalate to developers and then come back a month later and ask to retest, but some of this took multiple round trips. I was only using this on a side project, but definitely wouldn't consider them for anything critical, even if they are quite cheap.
"does not matter if this is an isolated edge case. Data loss for 15 months with poor support isn't something that can be waved off as 'edge case'. The fact I had to go to Reddit to get someone's attention for this is insane. By the way, I tried reaching out to various 'senior management' personnel via LinkedIn last year and no-one replied. Escalation requests via the Support thread ignored and declined. Very professional of you guys."
and that seems pretty damn reasonable of a reply. the fact they had to go to reddit is insane. a sarcastic "very professional of you guys" is pretty tame, given the situation, and not discrediting at all.
OP even actually responded to a similar comment in this reddit thread - look it up and tell me he wasn't reasonable. 15 months of fobbing him off, and once the case has been raised publicly the very senior guy _suddenly_ sees it as a priority despite it being just an edge case.
Really? Do you work for their PR or what?
That's basically how this whole industry works? How would you justify the outrageous cost of product management layers otherwise?
Not saying that this is the right way. But it's not a very surprising story, no matter which company (be it Google, Microsoft, OVH...).
Far from it. Most companies in the same space don't just ignore such significant problems. If they did, they wouldn't survive for very long.
Microsoft arbitrarily banning IPs/AS from its infrastructure with no recourse or acknowledgement of the issue (such as 365 lying about accepted mail traffic through SMTP)?
For paid products, denying or lying about there being any issue as serious as significant data loss for over a year like Bunny did, yes.
> Microsoft arbitrarily banning IPs/AS from its infrastructure with no recourse or acknowledgement of the issue (such as 365 lying about accepted mail traffic through SMTP)?
That sounds closer but still not comparable.
I've got a 250 and the ID, then the server discarded the email.
I am not sure but I do think that they might have a ticket system there too.
Either way both Hetzner and OVH would respond in better technical rigour
I used to like bunny but this ain't a good look :-(
https://www.youtube.com/watch?v=3m5qxZm_JqM
They've been ignoring the issue for over a year, yet it somehow took them only 4 hours after the reddit post was made to determine that it's "an isolated edge case", even though there's at least one other user reporting the same problem. The comment was clearly written with a focus on saving face rather than properly apologizing for the terrible support.
Someone recommended Bunny to me and I looked at it previously based on that but didn't have a use case then. But as someone who regularly recommends vendors for v.big CDN contracts I'm not impressed with their attempts to downplay things.
My experience was more positive. One time I had a minor issue with their storage where I couldn't replace a file or something. This was fairly early after the product launched. They fixed it and gave me free credit for reporting it.
I also tried Hetzner Object Storage. I love Hetzner, they're great, except for their Object Storage service, which is completely unreliable (errors, slowness, etc.). I'm surprised that Hetzner still hasn't retired that product until it's properly fixed.
My last chance is with Scaleway. Xavier Niel's products are always good, so fingers crossed...
At some point there was a bug in their payment system and I couldn't pay an invoice. The payment failed the first time due to lack of funds on my account (this initial failure was my fault). Once my account had funds, retrying failed with a vague error on Scaleway's side.
I reached out to support, and it was a known limitation: if your initial payment fails, you an only retry with a subset of their supported payment methods. I didn't have access to any of these payment methods, and I made it clear that I could pay, but only with the payment methods we'd been using thus far.
They kept sending me emails reminding me of this overdue bill and threatening to send collection agency, all while I kept exchanging messages with support complaining that their system gave an internal error.
The bill was, by the way, for less than €10. But a collection agency will charge you hundreds for the "collection" service.
They were aware of the error in their payment system, but it seems they had no intention of fixing it. After months of back and forth, they just cancelled the invoice.
The amount of time and stress in dealing with was ridiculous.
I have been using for sw for object storage for a while. I haven't had a a issue with them with object storage.
only problem I had was with a another product called cockpit(they ship metrics about object storage there), which they bundled with object storage product, which cannot be disabled.
This month I had a 12 euro surcharge, because they enabled something in cockpit by default a while ago and suddenly start to charging for it.
- Three copies
- Two types of media
- One off-site or offline
Admittedly 4 might be too many. But at some point I switched to r2 for the free public egress and deleting one of Aws or wasabi has never been a priority and I don't want to do it without putting in the time to quadruple check I'm not deleting anything important.
But at the very least id have 3. I'd hate to discover that my S3 was blown up in a war and then copying everything my HDD was the last straw that pushed an aging drive over the edge.
For my storage needs in the EU I'm using Upcloud's object storage. Very happy with them. Then a Bunny cdn zone if I need to share the files publicly.
I haven't had to contact Bunny's support so it's a bit disappointing to see this type of runaround from the support, but as an actual issue it seems pretty minor in the end.
I've been using Bunny for ~four years after looking at video hosting. In the four years we've saved approximately 50 000€ compared to if we'd gone with Cloudflare video streaming service and it's just been rock solid for us.
I upload all object storage stuff to bunny for live but also to backblace for backup.
I’ve always wanted to implement fail over client side for any asset over to bacblaze but seems like a lot of overhead
I expect the combination of
and means that detecting and dealing with the loss is substantially less work than moving away. [0] I expect that Management is fully aware of what's up and is making the call here.[0] "Just" add a retry if the post-upload verification step fails! Sure, it's slower, but it works, right??? mournful sob