By Simon Woodhead
Following our update on Friday, we have been monitoring the evolving DDoS situation and leveraging our industry reach to get the inside track from those on the front-line of assisting current victims. Based on this we have adapted our plans for handling such an attack against Simwood properties and some of those changes require customer action in order to benefit from them.
What follows is strategy and lacks specifics. We have to assume that our interop information, whilst closely guarded, is in the wrong hands. We will therefore not be modifying this to reflect any changes mentioned here. Instead, they will be communicated securely through our portals as a banner at the top and Community Slack. This also gives us the option of them being customer specific in the unlikely event of these being leaked.
In the event of a credible threat we will invoke these plans and will communicate that through the appropriate status page(s). We will not be giving our usual granular updates there however but will provide these through our Community Slack where we are more confident they will not be observed by bad actors.
Co-location / IP Transit customers
Customers co-located with us on static addressing are required to have equipment diversely in multiple Availability Zones . We will therefore be blackholing traffic to any targeted addresses to protect the network and other addresses utilised by that customer. We may route traffic to these address ranges through our DDoS scrubbing but our estimation is that this is unlikely given the risk of false positives.
Simwood Transit customers will, by definition, have their own IP addresses announced to Simwood and potentially others over BGP. We’d remind these customers of the tools already available, specifically:
- Blackhole an individual (/32) address to protect the customer (and Simwood) network. Traffic will be blackholed on the Simwood network and, where possible, before reaching the Simwood network.
- Scrub a block of addresses (minimum /24). This will force a block through DDoS scrubbing and is useful for non-UDP services. We anticipate that for SIP targeted attacks this could be counterproductive and blackholing is preferable.
Please note, we expect customers to use tools to manage their networks appropriately and we have adequate capacity on the Simwood network for them to run ports uncontended. However, in the event of a customer failing to do so and getting into a position of their port being saturated and a large volume of traffic being received onto the Simwood network (un-blackholed and un-scrubbed) we will shutdown BGP sessions to protect the network and other customers.
Lastly, customers using Simwood connectivity on static addressing will be handled in a similar way but we think it is unlikely that they are within the scope of this campaign and therefore our standard procedures apply.
Simwood Direct & Simwood Partner
We are making no announceable changes here and customers need take no action providing SRV DNS is used and the published address ranges are whitelisted. In the event of an attack we will scale customer facing SIP equipment and add new IP addresses as appropriate. Other aspects of our network management will ensure that bandwidth is not a concern here.
We have identified one service which cannot be optimally handled this way and that will be addressed early next week as the necessary development work has already been completed.
Simwood Wholesale
SIP/RTP
There are certain prerequisites in our existing interop that network operators using our wholesale services need to comply with:
- using SRV in DNS resolution of our fully qualified domain name for outbound traffic rather than hardcoding IP addresses
- Being meshed to all Availability Zones. This is implicitly done through SRV support (out) and appropriate ACL configuration (in/out) per our interop
We have banged on about the above, particularly SRV, repeatedly and this is an occasion where that will be invaluable. Those customers who open support tickets every time an A-record changes because they refuse to either use SRV or accommodate not doing so appropriately (per the interop) will find our response plan service affecting. Those compliant with the existing interop, we hope will not.
Rather than DDoS scrubbing SIP addresses, our plan is simply to blackhole any addresses targeted. Every service exists in every Availability Zone and interop compliant customers should seamlessly use any and all of them. We have existing capability on-net to do this dynamically on internal tooling which is faster and less error prone than trying to reconfigure ACLs on the fly or relying on DDoS scrubbing. Where an address is blackholed, this only affects customers reaching us over transit / public Internet; customers on-net and using appropriate private connectivity will be unaffected.
However, there is a scenario where all addresses of a given service across all availability zones are targeted simultaneously, we blackhole all, leading to a successful denial of service for those not directly connected – something we’ve warned about repeatedly in encouraging customers to connect privately.
To avoid this we have brought a further Availability Zone into service. Without giving too much away, this isn’t one we’ve created on the fly, but one which has existed for some time and not in our published interop.
This Availability Zone may contain one or many (as scaling dictates) addresses for outbound traffic which will not be communicated to you. They will however be added to SRV as appropriate. The range for media which is required to be added to any ACLs will be communicated securely through our portal and Community Slack. It is imperative that these are added to customer configurations (where required) to ensure continuity of service.
Inbound traffic for customers relies on published individual addresses which customers authorise on their SBCs. Any new addresses will be communicated as above. These addresses will only be used in the event of us risking blackholing all existing proxies and thus risking inbound service. It is imperative that customers have added these addresses to configurations (where required) to ensure continuity of service in this circumstance. We anticipate that we will only add one such address but may, if required, add more and may even communicate distinct addresses to distinct customers (through the portal) if we have doubts about this information being misused.
DNS
DNS is not handled on-net as we determined many years ago that (rather like e-mail) whilst we could do it on-net for free, we were better served engaging a specialist. We are very comfortable with the existing provision of mitigation capability in place there and do not envisage any changes.
Other services
Non-SIP services such as our API and portal lend themselves better to DDoS scrubbing and will therefore have any transit / public internet traffic passed through DDoS scrubbing. We will do this as a proactive measure on receipt of a credible threat, or before if we consider it appropriate.
We do not intend passing traffic received over private connectivity this way as we do not anticipate it being required. We will do so if required though.
***
We hope all the above proves unnecessary but want to ensure all customers are aware of our plans to avoid us chasing our tails under the heat of an attack. In that situation, we will of course rely on customers doing their part as well and not opening avoidable tickets or being aggressive to our team.
I’ve said a couple of times how proud I am of the Simwood Team and this is one of those moments where I’m reminded. They rock and it is my privilege to lead an incident call and be the dumbest person in the room. I hope this time next week you’ll agree with me in the unlikely event of you not doing so already (and reading this for voyeuristic pleasure!).
Lastly, whilst this is all about us, we’re here for our customers. It is highly likely some of our customers will fall victim to this campaign and we stand ready to help where we can. Do please lean on us if we can help.
