POE4ORBS is The Best Place to buy cheap poe Currency, Items and Power leveling service!

poe4orbs:PoE 3.14 Ultimatum Server Issues Fixing Progress

PoE 3.14 Ultimatum has been released for a few hours, but a large number of players have encountered server issues that are difficult to solve. GGG gave the latest news about when the server will be repaired normally.

 

Path of Exile Ultimatum

 

From official forum

Hey everyone,

It's been a long day but we wanted to put together a few thoughts while we have a moment waiting for our next server fix to build. This launch has been rough, to say the least. In this post, we plan to address both the ongoing technical realm stability issues and the conversation around streamers getting priority in the login queue. We are sorry that this is being addressed so late in the day - we have been giving the server issues absolute priority and haven't had time until now to write up this explanation.

Let's start with the technical issues.

Immediately upon launch of the league, we could see that the queue was running incredibly slowly. At the rate that it was emptying, it'd be at least two hours to get everyone into the game. The reason was that when players logged into their accounts, the server would migrate any previously un-migrated Ritual characters to Standard, which can take quite a lot of time to do on-demand (as much as three or four seconds per character in some cases). Users who had already logged in since Ritual ended were already migrated and were nice and fast. Normally, we run a "trickle migration" process in the background that performs this action on every account over the few days between the last league ending and the new one starting. Due to human error, this process was not run and hence the queue was unbearably slow to empty. (We have since codified this step into a QA checklist so that can't be trivially missed again in the future.)

We realised that a solution was to disable the Ritual-Standard migration entirely, which would result in the queue emptying very quickly but players would miss some Standard progress until we run it again later on. This solved the queue speed issue by around the one hour mark. At which point, the realm freaked out and dumped most of the players out, then continued to do this roughly every ten minutes or so for the rest of the day.

This wasn't good. At all. Aside from catastrophically ruining our launch day, it completely mystified us because we have been so careful with realm infrastructure changes. We thoroughly tested them internally, peer code reviewed them, alpha tested them, and ran large-scale load tests up to higher player capacities than we got on launch day. We even went so far as to deploy some of the database environment changes to the live realm a week early to get real user load on them just in case. But yet it still imploded hard on release.

I'll spare you the blow-by-blow of the hundred changes we have made over the last 12 hours, but we have been trying things one at a time in order of likelihood to fix the problem. There is one change we have been leaving for last (because it requires some downtime), but we have exhausted everything else we can think of, so we're trying that next. In the next 30-60 minutes after posting this, there will be roughly 30-60 minutes of hard downtime to make this change. We are optimistic that it stands a good chance of resolving the issue.

We will continue to work on this issue until the servers are working perfectly. We know the Path of Exile realm can handle this much load, it's just a matter of divining what subtle fuckery is causing the problem today.

Some players have also become concerned that when server issues occur, items are occasionally duplicated or destroyed when placed in a guild stash. This is a longstanding consequence of how our guild stashes work and generally isn't of much concern because players can't induce server problems and can't control whether the item is duplicated or destroyed. We are keeping a close eye on this of course.

So while this was all going on, we managed to also commit a pretty big faux pas and enrage the entire community by allowing streamers to bypass that really slow queue we mentioned. The backstory is that we have recently been doing some proper paid influencer marketing, and that involves arranging for big streamers to showcase Path of Exile to their audiences, for money (they have #ad in their titles). We had arranged to pay for two hours of streaming, and we ran right into a login queue that would take two hours to clear. This was about as close as you could get to literally setting a big pile of money on fire. So we made the hasty decision to allow those streamers to bypass the queue. Most streamers did not ask for this, and should not be held to blame for what happened. We also allowed some other streamers who weren't involved in the campaign to skip the queue too so that they weren't on the back foot.

The decision to allow any streamers to bypass the queue was clearly a mistake. Instead of offering viewers something to watch while they waited, it offended all of our players who were eager to get into the game and weren't able to, while instead having to watch others enjoy that freedom. It's completely understandable that many players were unhappy about this. We tell people that Path of Exile league starts are a fair playing field for everyone, and we need to actually make sure that is the reality.

We will not allow streamers to bypass the login queue in the future. We will instead make sure the queue works much better so that it's a fast process for everyone and is always a fair playing field. We will also plan future marketing campaigns with contingencies in mind to better handle this kind of situation in the future.

It's completely understandable that many players are unhappy with how today has gone on several fronts. This post has no intention of trying to convince you to be happy with these outcomes. We simply want to provide you some insight about what happened, why it happened and what we're doing about it in the future. We're very unhappy with it too.

In the meantime, we will continue to work on fixing the technical problems with the realm servers. We're hoping for a resolution to this as soon as possible and will keep you updated through edits to this post.


Anyone are welcomed to redistribute, copy, quote, reference or perform content in this site, if and only if he/she attributed the work
Related News
06
Dec

[Mayhem] Buy Cheap PoE Currency with Fast Delivery

Buy cheap mayhem currency in Poe4Orbs to help you defeat these powerful enemies. Poe4Orbs is the best place for you to buy poe currency, safe and fast, 24/7 online support.

10
Dec

I wish the community of Path of Exile had been smart

I mean sure, if you bought a ton of extra stash tabs and can physically hold that much poe currency, go ahead.

17
Oct

This is my honest feedback - Path of Exile Designers

This game has a unique feel and it is not easy to offer a self contained and well defined experience nowadays. Path of Exile is grim, compelling and harsh, in a good way.

17
Dec

Just a way from Grinding Gear Games to make you burn your poe currency

I love Path of Exile in the sense them Grinding Gear Games has made many good changes to the game along with it being free.

21
Oct

Frost wall spell was set to 100% chance to block your projectiles again in POE

Most cold projectiles shatter on impact ( arctic breath, cold shot), this would be the cold version of poe Orb of storm for lightning.

14
Jul

poe4orbs:PoE 3.11 Harvest All Challenges Guide

As always, PoE 3.11 has 40 challenges for players to unlock, and every 12 challenges unlocked will get a league reward, which are footprint, cloak, and portal. This guide aims to help players in completing those challenges.

Leave A Reply

Hot Payment Products

Poe4orbs Top News

17
Aug

poe4orbs:Path of Exile: Trial of the Ancestors

16
Aug

poe4orbs:New and Changed Gems in PoE Trial of the Ancestors

08
Apr

poe4orbs:POE 3.21 Crucible Unique