Peering & Transit - Global Transit

Summary

In order to be able to access all destinations on the Internet or to make your content available to the whole Internet, in a Autonomous Routing network Global Transit is required.

Global Transit provides worldwide routing, to and from all possible destinations on the public Internet over one link and BGP session. For Non-Stop redundancy at least two connections (preferable to different parties) are required.

Open Peering is reseller of Global Transit carrier Joint Transit, and offers their services as part of turn-key solutions. Joint Transit provides Global Transit services starting at 3,95 Euro per Mbit/s actual traffic per month.

Description

Provides routing to/from the whole world

A Global Transit service on the one hand provides your network with routing information describing how the entire Internet (currently over 210.000 networks, a full routing table) can be reached.

On the other hand the service advertises the IP Space you publish through to the rest of the Internet in the form of routing information, so everybody knows how to reach your network.

Includes Dutch routing: no Peering or NL Routing required anymore?

Global Transit covers all Internet destinations in the whole world. Therefor it inherently also includes access to all local (Dutch) destinations.

So yes, strictly speaking peering on a Internet Exchange, or using some form of partial transit like NL Routing is not required in order to be able to reach all local destinations when you have a Global Transit connection.

Peering and NL Routing however generally provide much faster (shorter paths) routing and substantially lower costs for traffic to and from local destinations. Therefor, for local traffic Global Transit is generally only used as means of emergency backup.

Requirements for connecting

In order to use Global Transit you need (1) a router(s) that supports the BGP protocol, (2) your own AS Number and (3) your own block of at least 256 consecutive IP Addressess.

Port or burst speed

Global Transit services are generally sold in the form of ethernet connections with a 10 Mbit/s, 100 Mbit/s, 1 Gbit/s or 10 Gbit/s port speed. In most cases the full port speed is available: the link can be filled up to the port speed during traffic bursts. In some cases the carrier sets a rate limit at some point lower then the port speed so no more traffic can be used then this agreed maximum speed.

Maximum usable traffic level

It is generally unwise to use more the 75% of the port speed on average, because from that point on the port is will already be filled 100% during brief sub-second bursts (congestion).

When more traffic if offered then can be handled by the port during a certain time period, traffic will be buffered temporarily until enough room on the port is available. Such buffers however last only a fraction of a second until they are filled up. For example a 64 Kbyte port buffer on a 100 Mbit/s port can be filled within 5 milliseconds. When the port buffer is full, traffic will be dropped. In higher speed ports the time before buffers are exhausted can be much shorter then this.

A 5 minute average graph of a port gives a value over a 300.000 millisecond time interval. Brief bursts of tens or hundreds of milliseconds are completely averaged out in such graphs, while such burst can easily overflow the port buffers (in the above example within 5 milliseconds) and cause packet loss.

When an average traffic graph shows over 75% usage, one should assume that buffers are exhausted regularly, and loss is already occurring, albeit maybe very modest and briefly. Even on ports with very low average traffic volumes, e.g. less then 5% usage, traffic loss can already incidentally be occurring when loaded with very brief but very high volume traffic bursts. Therefor 5 minute average traffic graphs are not a reliable way to ascertain that no packet loss due to congestion is occurring. Over provisioning port capacity is an effective way to prevent such problems as much/long as possible.

The 95% model for traffic measurement

The 95% model is a method to determine the traffic volume on a connection over a calendar month and is used my almost every Global Transit carrier for billing purposes:

  • Every 5 minutes two measurements (transmit and receive) are made of the total traffic that has passed over the port since the last measurement;
  • At the end of the calendar month all measurements (the highest of transmit and receive) of that month (generally 12*24*30=8.640) are lined up and sorted from high to low;
  • The highest 5% of the measurements (so generally 5% * 8.640 = 432) is dropped, and the next highest measurement defines the 95% traffic level on which the billing for that month is based.

Practically this means you can burst for (432 * 5 minutes / 60 =) +/- 36 hours per month without any impact on the monthly bill. The entire Internet is based on this model, and offering a different billing system (e.g. per total gigabyte of traffic per month) brings extra risk for a carrier which will result in a higher price. Therefor a 95% model will normally always be the lowest cost form of transit.

Billing

Generally Global Transit services are billed based on fixed monthly port fees and expected or committed traffic levels per quarter in advance. Then per month in rear these quarterly bills are corrected based on the actual traffic use in that month.

Traffic commitment

When you buy a Global Transit service with a certain traffic commitment, that means all traffic up to this commitment level is already included in the fixed monthly price. Any actual traffic over the commitment level is charged based on an agreed burst fee. Example:

  • You have a 10 Mbit/s commitment and actually use 14 Mbit/s. In addition to the fixed monthly fee (which includes the 10 Mbit/s) you pay for 14-10= 4 Mbit/s of burst traffic;
  • You have a 5 Mbit/s commitment and actually use 3 Mbit/s. You only pay the fixed monthly fee and do not pay any burst traffic.

Wirespeed - non-overbooked

Global Transit services are be default (unless otherwise stated by the carrier) non-overbooked services: during normal circumstances the port capacity you buy should alway be available to you for bursting and inherently not be shared with other customers.

They idea behind this model is that on the one hand you pay a monthly fee for the port itself or for a traffic commitment, which covers the costs for the carrier to keep burst capacity in its network available for you. And on the other hand if you use burst capacity for longer then 5% of the time (95%'s model) you pay for the burst traffic anyway.

Outbound route advertisement attracts incoming (pull) traffic

How you advertise your own IP routes to the world via the Global Transit service influences how the rest of the world send traffic to you, so traffic which in incoming or pull traffic for you.

If you announce your IP routes via multiple transit carriers or peer on an Internet Exchange or use an NL Routing service, parties in the rest of the world can have multiple options (paths) to reach you. They can use the routing information you provide in your various announcements to decide which path to use.

You probably want to influence the rest of the world into using a path that for you either has the lowest cost or performance. However, you cannot force how others send their traffic: it is their privilege to decide how they route traffic that is for them outgoing to the Internet.

Inbound routes are used for outgoing (push) traffic

You can decided for yourself how you use the routes you receive from your Global Transit carrier(s) and from other sources (peering on an Internet Exchange, NL Routing) in the decision how to send your outgoing (push) traffic to the Internet. It is you privilege to decide this, as long as you do not send traffic to a party that did not advertise you routing information for that destination. You probably will want to prefer to use the fastest and lowest cost (peering and NL Routing) paths if available, and for all other destinations use the global transit carrier with the shortest and fastest route path.

Service Level Agreement or Best-effort?

Because no single carrier controls the total end-to-end path between all internet destinations, it is technically impossible to make guarantees regarding the performance of Global Transit services that cover all destinations. Therefor Global Transit services are generally considered best-effort services: the carrier guarantees he will make his best possible effort to deliver your packets at their destination within reasonable performance criteria, but cannot guarantee such delivery.

Some Global Transit carriers do however provide Service Level Agreements which offer a money-back guarantees on performance of either the availability of the port and BGP session on their router, performance within the carriers network or performance up to specific points on the Internet, typically major Internet Exchanges. These service level agreements still provide only best-effort delivery since they do not guarantee service the other half of the way, from the Internet Exchange Point to the final destination.

Carrier quality: what is his carrier spread?

No Global Transit carrier covers all possible Internet destinations with his own network and customers. Even the very few largest carriers in the world, which do not buy transit from anyone themselves and handle all traffic via peering (so called Tier-1 networks or Top-Level carriers), do not cover more then 20% of the Global Internet.

Therefor every carrier will alway have to hand over at least 80%, but generally much closer to 100% of his traffic to other networks via his respective upstream transit carriers or peers in order to get the traffic delivered.

If eventually the traffic is handled via multiple different transit carriers anyway, it is wiser to use a wide spread of Global Transit carriers and handover the traffic close to the traffic source already. One can then select the shortest path to each destination from that mix of carriers. That improves performance, as routing paths will generally be shorter and more direct. Further it improves resilience of the transit as multiple paths are available directly from the beginning of the path instead of somewhere further up the line (handover point).

It is therefor wise, while selecting a Global Transit carrier, to check over how much and how diverse upstream paths that carrier spreads its service, as that is an indication of how short the paths of that carrier will be and how good its redundancy is.

If for example a carrier transports all US destined traffic from Europe over its own backbone to the US, and hands it over there, its network between Europe and the US is a single point of failure. If your carrier has a wide upstream spread directly from the point where you connect, that will improve your service.

Redundancy via dual-carrier routing: improve your own carrier spread

In analogy to the carrier spread of your carrier, it is very advisable to use multiple carriers yourself as well. It will give you shorter routing paths, but more importantly more redundancy. If one of your Global Transit carriers has a network problem, you still can route your traffic via one of the other transit services you have.

However, generally it is relatively expensive to have a more carriers, as the per Mbit/s cost of transit services is generally strongly influenced by how much traffic you do via (or commit with) a carrier. If you handle all your traffic via one carrier, you'll get better pricing then if you spread out the same volume over for example 5 carriers.

We do advise you to use 2 Global Transit carriers as this substantially improves your redundancy, while still limiting the cost sacrifices you make by spreading your volume as much as possible. If, while selecting upstream Global Transit carriers, you pay special attention to the carrier spread of each party, you can still maximize the total local carrier spread.

Redundancy via dual-session routing to your carrier

Most Global Transit carrier offer you a single BGP session with one of their routers. Some offer dual session routing: you get two separate BGP sessions to two different routers of the carrier. During maintenance or problems with one of the carriers routers, you will still have full routing available via the second session. Such a setup substantially improves availability of your Global Transit connection.

Redundancy via dual-port routing to your carrier

A further step in improving redundancy is using not one but two physical ports to your upstream global transit carrier. You can then connect both ports to separate routers on your side. In this setup, during maintenance on or problems with one of your routers or patchcable problems, you will still have full routing to this carrier available via the second port.

Of course if you have two Global Transit carriers and connect each to a different router in your own network, you already have full redundancy and dual-port routing for each carrier is not strictly necessary to achieve redundancy. Dual-port routing to one carrier should not be considered as an equivalent alternative for dual-carrier routing!

However, in a dual-carrier setup dual-port routing to individual carriers still has advantages, as your routing path diversity (and thus the use of shortest paths) stays available even during partial outages.

Redundancy via dual-datacenter routing

An even further, and more expensive step to redundancy is to install routers and purchase Global Transit services in two geographically dispersed datacenters. If there are power or cooling problems or dark fiber cuts in one datacenter, multiple carriers in the same datacenter might be impacted by that. The chances that a Global Transit carrier in an entirely different datacenter is also impacted are slim. Such a multi-datacenter does however require some form of reliable link (e.g. dark fiber ring) between both datacenters and/or some form of redundancy in the rest of your network (e.g. redundant webservers load balancing and backupping each other over the different datacenters).

Is your Carrier specialized on Global Transit?

Every party that itself has a full Internet routing table can in theory sell Global Transit. Because of that, there are a lot op parties which offer Global Transit, even if their primary business is in a completely different market. This includes Internet Access Providers, hosting companies, leased line and ethernet providers, etc.

One disadvantage of such a me-too service is that the network and internal support organization of such a company are not designed and scaled for supporting Global Transit services. Especially during network problems a lack of specialization and knowledge can be dangerous for the continuity and performance of your Internet connection. It is therefor essential to select a Global Transit carrier which is specialized, understands the product and local market and has a substantial local customerbase in Global Transit services already.

Changing Carrier is easy

As long a you use Provider Independent IP Addresses (withing contract duration and cancellation terms of course) you can always change to a different carrier easily at any point in time. With PI addresses there is nothing locking you in to one specific carrier nor any number portability issue. Especially if you have two carriers already, during the transition from one carrier to another, there should always stay at least one carrier available guaranteeing reachability.

Documentation on Global Transit

For more details about Global Transit, please have a look at the Internet Standard Wikipedia.

Pricing

Open Peering resells the products of Global Transit carrier Joint Transit, as part of turn-key solutions. Joint Transit has specific product groups aimed at the Wholesale market, Carriers, Providers and the Business market. For all products, prices and specification, please look at the Joint Transit website.

Conditions

Datacenters

The Global Transit products of Joint Transit are available at these datacenters.

At 10.000 Mbit/s the product is readily available at the euNetworks, GlobalSwitch, Interxion AMS5, Nikhef, SARA, Telecity1, Telecity2, Telecity3 (Redbus) datacenters. At other datacenters special pricing applies.

Includes port

Joint Transit connections are delivered including a dedicated port on the datacenter of choice (included in the price).

  • 100 Mbit/s ports are available on RJ45/UTP connector only;
  • 1 Gbit/s ports are offered on multimode fiber (SX/850nm) by default, but also available on singlemode fiber (LX/1310 nm) at an additional initial fee of € 500,-;
  • 10 Gbit/s ports are available on singlemode fiber (LR/1310nm) exclusively.

Does not include patchcable

The demarcation point of the service is the port on the Joint Transit switch on the datacenter. After an order is placed via Open Peering, in the order confirmation a Joint Transit port ID will be assigned. With that port ID the customer can order the patchcable from its equipment to the Joint Transit port with the datacenter directly.

Delivery time

Technically, delivery time of all products is max 10 working days, but generally we can do it (much) faster. Please reserve some time (a few days) after this order delivery to jointly thoroughly check on both sides if the connection is 100% okay before taking it into production.

Start production traffic at the first day of the month

Billing-wise, contracts are charged per full calendar month. If a customer would start using a port for production traffic during the last days of a month for example, we would still have to charge both the fixed and variable fees applicable for the productfee for the entire month, even if the port was not technically delivered and available for the first part of the month yet. We are forced to bill for full months only, as the rest of the Internet and our upstreams also charge in the same way.