Linux QOS HFSC: различия между версиями
Материал из noname.com.ua
Перейти к навигацииПерейти к поискуSirmax (обсуждение | вклад) (Новая: =HFSC= * http://ace-host.stuart.id.au/russell/files/tc/doc/sch_hfsc.txt * http://linux-ip.net/articles/hfsc.en/ <PRE> Tutorial ======== HFSC stands for Hierarchical Fair Service Curve. ...) |
Sirmax (обсуждение | вклад) (Полностью удалено содержимое страницы) |
||
| Строка 1: | Строка 1: | ||
| − | =HFSC= |
||
| − | * http://ace-host.stuart.id.au/russell/files/tc/doc/sch_hfsc.txt |
||
| − | * http://linux-ip.net/articles/hfsc.en/ |
||
| − | |||
| − | <PRE> |
||
| − | Tutorial |
||
| − | ======== |
||
| − | HFSC stands for Hierarchical Fair Service Curve. Unlike its cousins CBQ and HTB it has a rigorous mathematical foundation that delivers |
||
| − | a guaranteed outcome. In practice that means it does a better job than either CBQ or HTB, both of which are essentially best effort guesses at solving |
||
| − | |||
| − | what is a surprisingly complex problem. Both of them get it wrong in subtle ways which shows up as them not meeting their bandwidth and latency guarantees. |
||
| − | </PRE> |
||
| − | |||
| − | HFTC - это Иерархическая Честная Обслуживающая Кривая (что бы это ни значило). В отличии от своих двоюродных братьев CBQ и HTB у HFSC есть строгое математическое обоснование которое дает гарантированный результат. На практике это означает что HFSC делает работу лучше чем CBQ или HTB, которые по сути тратят услилия на решение неожиданно сложных проблем. Оба они (CBQ & HTB) делают это неправильно хитрыми способами что приводит к недополучению полосы пропускания или гарантированных задержек. |
||
| − | <PRE> |
||
| − | |||
| − | The Service Curve |
||
| − | ----------------- |
||
| − | Like CBQ and HTB, HFSC assumes the user has defined classes of traffic. One class might be "telnet traffic from customer A" which will be assured low latency, assuming it uses a low total bandwidth. HFSC expects you to use one of filters tc provides to decide which class each packet belongs to. |
||
| − | </PRE> |
||
| − | |||
| − | Как и CBQ & HTB, HFSC предпологает что пользователь определил классы трафика. Один класс может быть "траффик телнета для пользователя А" который гарантирует низкую задержку пока этот класс использует малую полосу пропускания. HFTC ожидает что вы будетет использовать фильты которые предоставляет tc для того что бы определить к какому классу отнести пакет |
||
| − | |||
| − | |||
| − | <PRE> |
||
| − | HFSC provides three types of service to each class, or to put it another way HFSC provides three knobs you can use to define the behaviour of each class: |
||
| − | </PRE> |
||
| − | |||
| − | HFSC предоставляет 3 типа обслуживания каждому классу, или если сказать это по-другому HFSC предоставляет рукоятки которые можно |
||
| − | использовать для определния поведения каждого из классов: |
||
| − | |||
| − | <PRE> |
||
| − | rt Real time. This gives a class a hard guarantees about its real time speed. |
||
| − | </PRE> |
||
| − | |||
| − | Real time. Этот флаг (ручка? =) ) дает классу жесткие гарантии о его real time speed. (примечание: гарантированная скорость?) |
||
| − | |||
| − | <PRE> |
||
| − | ls Link share. If the link has data queued up to be sent (ie it is backlogged), its share of the links capacity is in proportion to this speed. Every class must have a real time or link share definition. |
||
| − | </PRE> |
||
| − | Link share. Если у линка есть данные поставленные в очередь на отправку, емкость линка распределяется пропорционально этой скорости. |
||
| − | Каждый класс должен быть определен как rt или ls (примечание: не помешал бы пример!) |
||
| − | |||
| − | |||
| − | <PRE> |
||
| − | ul Upper limit. The maximum speed link share can send at. It can only used if the class has a link share defined. |
||
| − | </PRE> |
||
| − | Upper limit. Максимальная скорость на котоой ls класс может посылать данные (примечание: ceil?) |
||
| − | |||
| − | <PRE> |
||
| − | The two letter codes above, "rt", "ls", and "ul" is how you identify these three types of service to the tc command. There is |
||
| − | a fourth one: "sc". "sc" is a shorthand for setting both "rt" and "ls" to the same thing. |
||
| − | </PRE> |
||
| − | Двухбуквенные коды выше "rt", "ls" и "ul" это описание как определить эти три уровня обслуживания для команды tc. Существует так же четвертый код: "sc". "sc" это сокращение от "setting both" - установка обоих "rt" и "ls" для одного (класса?) |
||
| − | |||
| − | <PRE> |
||
| − | The speed allocated to each service is specified as a normal speed and an optional burst speed: |
||
| − | |||
| − | SERVICE [m1 BPS d SEC] m2 BPS |
||
| − | </PRE> |
||
| − | Скорость выделенная для каждого сервиса определяется как нормальная скорость и опционально - burst speed |
||
| − | <PRE> |
||
| − | Where: |
||
| − | |||
| − | SERVICE "ls", "rt", "ul" - ie whether this specifies the link sharing, real time or upper limit service. |
||
| − | |||
| − | m2 BPS This is the steady state speed, eg "m2 10kbps" means the steady state allowance for this service class is 10 kilo bytes/sec. |
||
| − | |||
| − | m1 BPS d SEC The burst speed. "m1 20kbps d 10ms" means this class is allowed a burst speed of 20kbps for 10ms, then it must run at its steady state speed. |
||
| − | </PRE> |
||
| − | Где |
||
| − | * SERVICE "ls" "rt" "ul": - определяет тип сервиса |
||
| − | * m2: это "скорость устойчивого состояния", например "m2 10kbps" означает разрешение устойчивого состояния этого класса в 10 кбит в сек. Измеряется в bps |
||
| − | * m1 это burst speed "m1 20kbps d 10ms" означает что классу разрешено burst speed 20kbps на 10ms а затем должен использовать "скорость устойчивого состояни" |
||
| − | <PRE> |
||
| − | In the specification and tc call this is called a "service curve".It is a curve in the same sense that any mathematical function can be plotted as a curve. In the paper this curve is used as the basis for mathematical proofs about the performance of the scheduler. Unless you also want to do mathematical proofs avoid thinking avoid thinking of specification as a curve. It is a speed with an optional latency specification - nothing more. |
||
| − | </PRE> |
||
| − | В спецификации и вызовах tc это называют "сервисной кривой". Это кривая имеет тот же смысл что и математическая функция которая может быть изображена как кривая. Эта кривая используется как математическое доказательство производительности планировщика. Если вы не собираетесь делать математические доказательства не думайте о спецификации как о кривой - это скорость с дополнительной спецификацией задержки и ничего более. |
||
| − | |||
| − | <PRE> |
||
| − | (There is another syntax supported which is documented in the reference section below - "umax BYTES dmax SEC rate BPS". "rate BPS" is identical in meaning to "m2 BPS". The definition of "m2 BPS" does *not* always express specify a burst rate and time. I have not found any other documentation of what it can yield, probably because it is so complex to explain. Such complexity is best avoided - don't use "umax ... dmax ...". The original terms came from the paper, but tc didn't follow the paper's definitions of them to the letter.) |
||
| − | </PRE> |
||
| − | ( |
||
| − | Существует другой синтаксис который описан в секции ниже: "umax BYTES dmax SEC rate BPS". |
||
| − | * "rate BPS" соответвует "m2 BPS" |
||
| − | * "m2 BPS" не всегда выражает burst rate и время. Я не нашел другой документации что это означает - вероятно это сложно пояснить. Таких сложностей лучше всего избежать - не используйте "umax ... dmax ...". Оригинальные термины пришли с бумаги но tc не следовал этим определениям. |
||
| − | ) |
||
| − | |||
| − | <PRE> |
||
| − | Burst and Latency |
||
| − | ----------------- |
||
| − | |||
| − | The term "burst" hides more than it reveals. Yes, it specifies bursts, but it is not used to control bursts of data. It is used to control latency. This is not some minor quibble about naming conventions. Wrapping your head around this is fundamental to understanding HFSC. |
||
| − | </PRE> |
||
| − | Термин burst скрывает больше чем поясняет. Да он определяет burst но он не используется для burst данных а используется для контроля latency. |
||
| − | Это не каламбур - это фундамент для понимания HFSC |
||
| − | <PRE> |
||
| − | But before going on, there is an even more important corollary to it being about latency. Most traffic doesn't care about latency. Web traffic doesn't, email doesn't. The two classes of traffic it might matter to are real time (like VOIP and NTP), and interactive traffic (like telnet, ssh, VNC, RDP and gaming). If you don't care about these do yourself a favour: don't use burst and skip this |
||
| − | section entirely. |
||
| − | </PRE> |
||
| − | Прежде чем продолжить есть еше более важное замечание касающееся latency. Большая часть трафика не чувствительна к задержкам. Например web и email - не чувствительны. Real time имеет значение для VOIP NTP и интерактивного трафика - если вы не беспокоитесь об этом не используйте burst. |
||
| − | <PRE> |
||
| − | And a caveat. Any effort to control latency will fail abysmally (as in be a complete and utter waste of your time and your CPU's cycles) unless class doing the controlling is the slowest link in the chain. The moment any hop between source and destination starts buffering packets HFSC's efforts to constraint latency will be overwhelmed by delays caused by the buffering. In case it isn't obvious, this means you can never control ingress latency. |
||
| − | </PRE> |
||
| − | И напротив любые попытки контролировать задержки не пройдут. Это будет пустой тратой времени и циклов процесора если класс не контролирует самый едленный линк в цеочке. В момент когда хоп (роутрер) начинает буферизировать пакеты HFSC перестает контролировать задержку. Это неявно означает что вы никогда не сможете контролировать входную задержку (эту секцию написал Капитан очевидность) |
||
| − | <PRE> |
||
| − | To see how the burst speed effects latency, imagine we have two |
||
| − | classes, say ssh and web, which we are happy to devote 50% of the |
||
| − | link to over the long term. |
||
| − | |||
| − | When the link is idle the HFSC scheduler is given a 1499 byte web packet and a 1500 byte ssh |
||
| − | packet to send. HFSC must decide which one to send one first, and |
||
| − | after it does the other will be a backlog, awaiting it's turn to |
||
| − | be sent. |
||
| − | It is only if the link will become backlogged that the |
||
| − | burst speed comes into play, and even then it is only for a |
||
| − | short while - the length of time being specified by the SEC in |
||
| − | "m1 BPS d SEC". In effect all it does is determine which of these |
||
| − | packets get sent first. And that is how we get to latency - |
||
| − | because whichever is sent first, ssh or web, will be perceived to |
||
| − | have the lower latency. |
||
| − | </PRE> |
||
| − | Что бы увидеть как burst speed отражается на задержке представьте у нас есть 2 класса, скажем ssh и web которым мы рады выделить 50% полосы пропускания на длительный срок. |
||
| − | <BR>Когда линк простаивает и планировщик HFSC получает на отправку 1499 байтовый пакет от web и 1500 байтовый пакет от класса ssh и HFSC должен решить который из пакетов отправить первым и после отправки другой пакет будет отложен ожидая своей очереди на отправку. |
||
| − | |||
| − | Только если есть отложенные данные burst speed вступакет в игру и даже тогда только на кроткое время - на время которое было определено в SEC в "m1 BPS d SEC". Эфект который жто дает - это определение какой иэ этих пакетов был отправлен первым. И это как мы получаем задержку потому что (пакет) отправленный первым будь то ssh или web будет иметь меньшую задержку. |
||
| − | |||
| − | <PRE> |
||
| − | To figure out which packet to send HFSC calculates how long each will take to send, and sends the one that will finish sending first. The twist is it doesn't use link's raw speed to calculate this time, it uses the speed for the service. All three services (link share, real time and upper limit) do their calculations independently. |
||
| − | </PRE> |
||
| − | Что бы определить какой пакет отправлять HFSC вычисляет как долго займет отправка каждого и отравляет тот, отправка которого завершитья раньше. |
||
| − | Обман в том что скорость линка не используется для вычисоения этого времени, а используется скорость определнная для сервиса. Все три сервиса (link share, real time и upper limit) делают вычисления независимо |
||
| − | <PRE> |
||
| − | Some examples will hopefully make this clear. Lets allocate web to classid 1:1, and ssh to classid 1:2, and say we have a 1mbps link. |
||
| − | </PRE> |
||
| − | Несколько примеров которые надеюсь сделают это все более понятным. Давайте создадим класс web (1:1) и ssh (1:2) и скажем что у нас 1мбитный линк (тут пояснить!!!!) |
||
| − | <PRE> |
||
| − | Example 1. |
||
| − | |||
| − | tc class add dev eth0 parent 1:0 classid 1:1 hfsc \ |
||
| − | ls m2 500kbps # web |
||
| − | tc class add dev eth0 parent 1:0 classid 1:2 hfsc \ |
||
| − | ls m2 500kbps # ssh |
||
| − | </PRE> |
||
| − | Примечание: |
||
| − | |||
| − | <PRE> |
||
| − | |||
| − | We haven't set a burst speed, so the time it takes to send the |
||
| − | packets is: |
||
| − | |||
| − | web 1499 / 500000 = 0.002998 sec |
||
| − | ssh 1500 / 500000 = 0.003000 sec |
||
| − | |||
| − | So in this example web takes the shortest time to send, so it |
||
| − | will it be send first and have the lower latency. |
||
| − | </PRE> |
||
| − | |||
| − | |||
| − | <PRE> |
||
| − | Example 2. |
||
| − | |||
| − | Many would prefer ssh traffic to have lower latency than web. |
||
| − | This will achieve it: |
||
| − | |||
| − | tc class add dev eth0 parent 1:0 classid 1:1 hfsc \ |
||
| − | ls m1 100kbps d 1.5ms m2 500kbps # web |
||
| − | tc class add dev eth0 parent 1:0 classid 1:2 hfsc \ |
||
| − | ls m1 900kbps d 1.5ms m2 500kbps # ssh |
||
| − | |||
| − | Since this is the start of a backlog period it also marks the |
||
| − | start of a bust period, so the bust speed in in effect. Thus |
||
| − | we do that same calculation as in Example 1, but with burst |
||
| − | speeds: |
||
| − | |||
| − | web 1499 / 100000 = 0.014990 sec |
||
| − | ssh 1500 / 900000 = 0.001667 sec |
||
| − | |||
| − | In this scenario ssh will be sent first and thus appear to have |
||
| − | the lower latency, which is what we wanted. In fact it because |
||
| − | of the ratios chosen (100kbps for web versus 900kbps for ssh) a |
||
| − | ssh packet would have to be 10 times larger than a web packet |
||
| − | before it got sent second. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | The burst must be long enough to send the entire packet. That is |
||
| − | what determined the 1.5ms above. Our link runs at 1mbps, so it |
||
| − | takes (1500 / 1000000 = 1.5ms) to send a Maximum Transmit Unit |
||
| − | (MTU) sized packet, assuming the MTU is 1500 bytes. |
||
| − | </PRE> |
||
| − | |||
| − | |||
| − | <PRE> |
||
| − | Also notice the burst speed for web doesn't look like a "burst" |
||
| − | at all. It is far slower than the steady state speed web normally |
||
| − | receives. This is because it really *is* being used to control |
||
| − | latency, not to specify a burst. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | Example 3. |
||
| − | |||
| − | That worked, but not as well as you hoped. You recall TCP can |
||
| − | send multiple packets before waiting for a reply and you would |
||
| − | like them all sent before web gets to send it's first packet. |
||
| − | You also estimate you have 10 concurrent users all doing the same |
||
| − | thing. From a packet capture you decide 4 sending packets before |
||
| − | waiting for a reply is typical. |
||
| − | |||
| − | 10 users by 4 packets each means 40 MTU sized packets. Thus you |
||
| − | must adjust the burst speed, so ssh gets 40 times the speed of |
||
| − | web and you must allow for 40 MTU sized packets in the burst |
||
| − | time: |
||
| − | |||
| − | tc class add dev eth0 parent 1:0 classid 1:1 hfsc \ |
||
| − | ls m1 24kbps d 60ms m2 500kbps # web |
||
| − | tc class add dev eth0 parent 1:0 classid 1:2 hfsc \ |
||
| − | ls m1 975kbps d 60ms m2 500kbps # ssh |
||
| − | |||
| − | Note that 975:24 is 40:1, and (40 * 1500 / 1mbps = 60). |
||
| − | |||
| − | If a class's burst speed is truly a burst (ie allowed it to send |
||
| − | faster than it's normal steady state speed), it will only be |
||
| − | allowed to do it if it has been sending at less than it's steady |
||
| − | state speed for long enough to accrue the headroom required for the |
||
| − | burst. To put it another way, a class will not be allowed a burst |
||
| − | if over the long term it would mean it is running over it's steady |
||
| − | state speed. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | On the other hand if a classes "burst" is really a "go slow" than |
||
| − | (like web's) it can be have its bandwidth pinched at any time by a |
||
| − | class (like ssh) that is allowed a burst, thus giving it truly bad |
||
| − | latency. In this sense ssh's good latency doesn't come for free. |
||
| − | It gains it at the expense of web having bad latency. This will |
||
| − | always be true, and is conceptually no different to the trade off |
||
| − | you make for bandwidth. If you guarantee web good bandwidth over |
||
| − | the long term, then some traffic (eg email) must be getting bad |
||
| − | bandwidth when web is using it. Similarly web's latency will be |
||
| − | bad when ssh is making use of its latency advantage. However, |
||
| − | because HFSC guarantees that ssh will not exceed it's steady state |
||
| − | bandwidth over the long term web will still average at least the |
||
| − | bandwidth you allocated it. In the HFSC paper, they say the |
||
| − | latency and bandwidth specifications are decoupled, meaning over |
||
| − | the long term one has no effect on the other. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | Finally, since HFSC favours packets that can be sent quickly it |
||
| − | favours sending small packets before large ones. By happy |
||
| − | coincidence this is generally what you want. VOIP, DNS, NTP, TCP |
||
| − | startup and ACKs are all small. Since the advantage you get from |
||
| − | giving a class a high burst is measured in milliseconds, not |
||
| − | seconds, specifying a burst for link sharing often isn't worth the |
||
| − | effort. |
||
| − | </PRE> |
||
| − | |||
| − | |||
| − | <PRE> |
||
| − | The meaning of "real time" |
||
| − | -------------------------- |
||
| − | |||
| − | tc-hfsc's web page follows the terminology in the paper and says |
||
| − | "real time" gives guaranteed bandwidth. Web pages talking about |
||
| − | HFSC take this to imply link share does not give guaranteed |
||
| − | bandwidth. This is true, but misleading. The "guarantees" given |
||
| − | by the alternatives (like CBQ and HTB) are no better. |
||
| − | </PRE> |
||
| − | |||
| − | |||
| − | <PRE> |
||
| − | The real time gives you something nothing else does: a hard |
||
| − | deadline. Link share's will latency will on occasion be out by a |
||
| − | 100's of milliseconds, very occasionally seconds. But notice we |
||
| − | are talking seconds and milliseconds here. If you do not have |
||
| − | traffic that cares about millisecond delays then you don't care |
||
| − | about real time. There are very few traffic classes that do care - |
||
| − | only VOIP and maybe NTP spring to mind. In particular if you were |
||
| − | a satisfied user of HTB or CBQ, you don't care about real time. If |
||
| − | you don't care about real time do yourself a favour: don't use it, |
||
| − | and stop reading this section. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | Real time's guarantees do not come for free, nor are they what they |
||
| − | seem. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | The first issue is in order to give hard real time guarantees the |
||
| − | real time service must take into account all data a class has sent |
||
| − | since the link was brought up. If the class has has exceeded its |
||
| − | real time steady state speed in the past (probably by using |
||
| − | bandwidth when no one else wanted it) then it can be denied *any* |
||
| − | bandwidth by the real time class until it gets under its long term |
||
| − | steady state speed. In practice this probably won't be an issue, |
||
| − | but being penalised for using bandwidth no one else wanted it |
||
| − | usually thought of as "unfair" (yes this is a well defined term |
||
| − | when dealing with quality of service), and thus the real time is an |
||
| − | unfair allocator of bandwidth. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | The second issue is in order to provide it's guarantees, real time |
||
| − | may have keep the link idle in case a high priority packet needs to |
||
| − | be sent in order to maintain latency guarantees. Ego there may be |
||
| − | times when a packet is waiting to be sent, but the link is idle, |
||
| − | thus the link will be artificially constrained to below its rated |
||
| − | capacity. A service that always takes advantage of the links rated |
||
| − | capacity is called work preserving. Link share is work preserving. |
||
| − | Real time isn't. There is a corollary to this. Because real time |
||
| − | delays sending packets it's accuracy can depend on the resultion of |
||
| − | the kernels timers. For example, if your hardware + kernel only |
||
| − | supprts a 100hz timer, the best latency resolution you can get is |
||
| − | 10 milliseconds. Common modern hardware provides very accurate |
||
| − | timers, but it's worth your while checking if you are taking the |
||
| − | road less travelled. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | The third issue is real time provides a hard latency guarantee in |
||
| − | the sense that latency is bounded to an absolute maximum. However |
||
| − | that absolute maximum isn't necessarily what you specified when you |
||
| − | said "m1 12500 d 2ms". You might expect that guarantees a 250 |
||
| − | byte packet will be sent within 2ms. It doesn't, unless you do not |
||
| − | use link share or put any other qdisc between HFSC and the device. |
||
| − | Recall that real time will deliberately hold the line idle in order |
||
| − | to give it's guarantee. The issue is link share will sneak in and |
||
| − | send a packet over that idle link while real time isn't watching. |
||
| − | In practice this means real time's latency guarantees can be out by |
||
| − | the time it takes to send one MTU sized packet. So the guarantee |
||
| − | isn't 2ms. It's (2ms + time to send MTU sized packet). You can of |
||
| − | course compensate for this by specifying the time as (2ms - time to |
||
| − | send MTU sized packet), provided your link is fast enough to send |
||
| − | MTU sized packet + 250 bytes within 2ms. |
||
| − | </PRE> |
||
| − | |||
| − | </PRE> |
||
| − | The fourth issue is real time says nothing about what to do with |
||
| − | the links spare capacity. If you don't use link share as well, |
||
| − | the guaranteed minimum also becomes a maximum. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | The fifth issue is unlike link share, the bandwidths you give |
||
| − | to real time must be accurate, and their sum must be below the |
||
| − | links capacity. This means the sum of the burst speeds must be |
||
| − | below the links capacity, and the sum of the steady state speeds |
||
| − | must also be below the links capacity. Ensuring this if don't you |
||
| − | use the same value for "d" or "dmax" in every class requires |
||
| − | solving multiple linear equations. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | The sixth issue is if you use up all your bandwidth with real time |
||
| − | guarantees, then when the link is backlogged link share will get |
||
| − | no share, as in will never send a packet and all that goes along |
||
| − | with that - like TCP connections timing out and dieing. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | In summary, real time good for one thing: guaranteeing hard |
||
| − | latencies. So specifying a real time service without a burst time |
||
| − | is a waste of time. The converse is also true: real time is |
||
| − | absolutely useless for anything but guaranteeing hard latencies. |
||
| − | Everything else is better done using link share. |
||
| − | </PRE> |
||
| − | |||
| − | |||
| − | <PRE> |
||
| − | Using link share and upper limit |
||
| − | -------------------------------- |
||
| − | |||
| − | Under link share the speed a backlogged class can send depends on |
||
| − | ratio of its speed to other backlogged classes. (If class isn't |
||
| − | backlogged then by definition it's sending packets as fast as it |
||
| − | wants, so link share doesn't need to arbitrate on it's behalf.) |
||
| − | Link share does one thing: when the underlying device says its |
||
| − | ready to send a packet, it looks at what classes that is furtherest |
||
| − | away from the proportion of the link it should have, and sends its |
||
| − | packet. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | An example. Lets say we have classes A, B and C, who have link |
||
| − | share specifications of: |
||
| − | |||
| − | class A: hfsc ls rate 10kbps |
||
| − | class B: hfsc ls rate 20kbps |
||
| − | class C: hfsc ls rate 30kbps |
||
| − | |||
| − | A and B are sending packets as fast as they can, so they will be |
||
| − | backlogged. C is idle. The ratio of the speeds ratio of the |
||
| − | backlogged classes, A and B, is 10 to 20, ie 1 to 2. So link share |
||
| − | ensures for every byte A sends, B sends 2. |
||
| − | |||
| − | It's so simple it's important to note what this calculation doesn't |
||
| − | depend on: |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | A. It doesn't depend on the speed of the underlying link. Link |
||
| − | share doesn't need to know what that speed is, and the numbers |
||
| − | you give it need have no relationship to that speed. Only the |
||
| − | ratios matter. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | B. This implies link share is immune to link speed changing during |
||
| − | the day, as real internet connections often do. |
||
| − | </PRE> |
||
| − | |||
| − | <PRE> |
||
| − | C. Link share doesn't care who sends the packets - it or real |
||
| − | time. They are all counted the same way. |
||
| − | </PRE> |
||
| − | |||
| − | |||
| − | There are some complications as well: |
||
| − | |||
| − | A. Link share will not give a class more than the upper limit. |
||
| − | Real time does not respect upper limit, but since link share |
||
| − | counts what real time sends going over the upper limit using |
||
| − | real time will be penalised later by link share. |
||
| − | |||
| − | B. Link share only gets what's left over after real time sends |
||
| − | it's packets. To link share the bandwidth used by real time |
||
| − | looks like the link speed varying a bit faster than it would |
||
| − | otherwise. |
||
| − | |||
| − | C. A class is allowed to exceed its steady state speed by using |
||
| − | its burst speed when it first becomes backlogged. Once the |
||
| − | burst allowance is exhausted it can not exceed the steady state |
||
| − | speed until the backlog has been completely cleared. |
||
| − | |||
| − | And there is one negative. Link share will send packets as fast as |
||
| − | the underlying device will let it. If the underlying device isn't |
||
| − | the slowest thing between it and the destination link share won't |
||
| − | be the thing controlling what packet gets sent next - it will |
||
| − | be whatever scheduler is running on the slowest device. Typically |
||
| − | on a Linux box the device HFSC is controlling is an Ethernet NIC |
||
| − | that runs at giggabits per second. The slowest thing is probably |
||
| − | the thing it is connected to - the modem, so unless do something |
||
| − | the modem will managing traffic, not HFSC. If your Linux box isn't |
||
| − | controlling an Ethernet NIC its most likely attempting to schedule |
||
| − | traffic on a virtual device. Under Linux virtual network devices |
||
| − | like tunnels, ifb (used for ingress scheduling), and tun/tap |
||
| − | devices processing their packets as soon as they receive them. To |
||
| − | a scheduler they appear to be infinitely fast. This means on a |
||
| − | typical Linux setup not only will link share have absolutely no |
||
| − | effect on the outgoing traffic, it's propensity to flood the link |
||
| − | will destroys any chance real time had at controlling latency. |
||
| − | |||
| − | To fix this, you must limit the rate link share can send using |
||
| − | upper limit. Remember, for link share to be effective the limit |
||
| − | must be lower than the slowest link between the source and |
||
| − | destination. Upper limit has one other use - constraining a |
||
| − | customer to the bandwidth they paid for. |
||
| − | |||
| − | Finally, link share can't give hard latency guarantees. So its |
||
| − | burst numbers don't mean "send an ssh packet within 10 |
||
| − | milliseconds". Instead they mean "if ssh has been quiet for a |
||
| − | while, briefly increase the ratio at which ssh can send packets, |
||
| − | so it gets better latency than competing classes". That said, |
||
| − | if you follow these rules link share will get reasonably close |
||
| − | to the latency you ask for: |
||
| − | |||
| − | a. For classes that use real time, you use exactly the same |
||
| − | link share specification. Recall "sc" does this. |
||
| − | |||
| − | b. The total bandwidth you allocate for link share doesn't |
||
| − | exceed the links capacity or upper limit. This must be |
||
| − | true for both burst and steady state speeds. |
||
| − | |||
| − | |||
| − | The Class Hierarchy |
||
| − | ------------------- |
||
| − | |||
| − | Like HTB and CBQ, HFSC allows you to create a hierarchy of classes. |
||
| − | This is an example: |
||
| − | |||
| − | +1:0---------+ |
||
| − | | qdisc | |
||
| − | | 1500kbit | |
||
| − | +------------+ |
||
| − | / \ |
||
| − | +-1:10------+ +-1:20------+ |
||
| − | | User A | | User B | |
||
| − | | 50% share | | 50% share | |
||
| − | | Unlimited | | 1000kbit | |
||
| − | +-----------+ +-----------+ |
||
| − | / \ |
||
| − | +-1:11-------+ +-1:12---------+ |
||
| − | | Web | | VOIP | |
||
| − | | 100% share | | 20ms Latency | |
||
| − | | Unlimited | | 100kbit | |
||
| − | +------------+ +--------------+ |
||
| − | |||
| − | Intuitively this does the following: |
||
| − | |||
| − | - User A can use all of the links bandwidth. |
||
| − | - All User A's spare bandwidth goes towards web browsing. |
||
| − | - User A has VOIP which must have low latency. |
||
| − | - User B can use a maximum of 1000kbit. |
||
| − | - User B gets 50/50 share of a congested link. |
||
| − | |||
| − | Up till now we have been discussed the leaf nodes sharing a common |
||
| − | parent (inner node) interact. This is how the hierarchy effects |
||
| − | each type HFSC service: |
||
| − | |||
| − | ls A parent enforces its link sharing allocation on its |
||
| − | children. In other words the siblings get to fight over |
||
| − | the proportion of the shared bandwidth allocated to their |
||
| − | parent, and the parent and it's siblings get to fight over |
||
| − | whatever their ancestors link sharing allocations are. |
||
| − | |||
| − | rt The hierarchy does not effect the bandwidth allocated by the |
||
| − | real time service any way. rt specifications for inner nodes |
||
| − | are ignored. |
||
| − | |||
| − | ul A parent enforces its upper limit allocation on its children. |
||
| − | In other words the sum of the bandwidth used by children's |
||
| − | link sharing service will never exceed any of their ancestors |
||
| − | upper limits. |
||
| − | |||
| − | In practice, for most devices, you will need one class enforcing an |
||
| − | upper limit so link share doesn't flood the link and thus destroy |
||
| − | HFSC's ability to schedule. That class will be the root, of |
||
| − | course. |
||
| − | |||
| − | With those definition in hand it this is how the above hierarchy |
||
| − | is implemented: |
||
| − | |||
| − | # |
||
| − | # Save typing when creating classes. |
||
| − | # |
||
| − | ca() { |
||
| − | par=$1; class=$2; shift 2 |
||
| − | tc class add dev eth0 parent 1:$par classid 1:$class hfsc "$@" |
||
| − | } |
||
| − | # |
||
| − | # Create the qdisc. Send unclassified traffic to 1:11. |
||
| − | # If "default" isn't given unclassified traffic is dropped. |
||
| − | # |
||
| − | tc qdisc add dev eth0 root handle 1:0 hfsc default 11 |
||
| − | ca 0 1 ls m2 1500kbit ul m2 1500kbit # Stop link flooding |
||
| − | # |
||
| − | # Create classes representing each user. |
||
| − | # |
||
| − | ca 1 10 ls m2 750kbit # User A |
||
| − | ca 1 20 ls m2 750kbit ul m2 1000kbit # User B |
||
| − | # |
||
| − | # User A's web traffic. |
||
| − | # |
||
| − | ca 10 11 ls m2 1500kbit # User A web |
||
| − | # |
||
| − | # VOIP. We said VOIP uses 100kbit, and required 20ms latency. |
||
| − | # We also know that HFSC can break its rt latency guarantees |
||
| − | # by one MTU packet. Assuming MTU is 1500 bytes, it takes: |
||
| − | # |
||
| − | # 1500 [byte] * 8 [bits/byte] / 100000 [bits/sec] = 12 msec |
||
| − | # |
||
| − | # to send an MTU sized packet. That means we must send the |
||
| − | # VOIP packet within 8ms (= 20ms - 12ms) to meet the latency |
||
| − | # requirements. |
||
| − | # |
||
| − | # The requirements didn't say how big a VOIP packet is, but given |
||
| − | # it has to send a new packet every 20ms the biggest it could be |
||
| − | # is: |
||
| − | # 100000 [bit/sec] / 8 [bit/byte] * 0.02 [sec/pkt] = 250 byte/pkt |
||
| − | # So, we have 8ms to send 250 bytes, which works out to be: |
||
| − | # 250 [bytes] * 8 [bit/byte] / 0.008 [sec] = 250kbit |
||
| − | # |
||
| − | ca 10 12 rt m1 250kbit d 8ms m2 100kbit # User A VOIP |
||
| − | |||
| − | |||
| − | |||
| − | Reference |
||
| − | |||
| − | ========= |
||
| − | |||
| − | Qdisc Parameters |
||
| − | ---------------- |
||
| − | |||
| − | default <CLASSID> |
||
| − | Send unclassified packets to <classid>. If not supplied |
||
| − | unclassified packets are dropped. |
||
| − | |||
| − | Class Parameters |
||
| − | ---------------- |
||
| − | |||
| − | ls <SERVICE-CURVE> |
||
| − | Apportions how much of a backlogged link's spare capacity |
||
| − | should be allocated to this class and its children if a packet |
||
| − | isn't being sent under the rt <SERVICE-CURVE>. The bandwidth |
||
| − | allocated by <SERVICE-CURVE> of all sibling classes wanting to |
||
| − | share the excess capacity is summed, then the ratio of the |
||
| − | links capacity after real time to the total bandwidth consumed |
||
| − | this class and all it's offspring is equal to the ratio the |
||
| − | classes <SERVICE-CURVE> bandwidth to the sum. When calculating |
||
| − | <SERVICE-CURVE>, <BURST-SEC> is measured from the last time |
||
| − | this class was backlogged, ie had packets waiting to be |
||
| − | transmitted. Every leaf class must have a ls or rt (or both) |
||
| − | <SERVICE-CURVE> specified. Every inner class must be given an |
||
| − | ls <SERVICE-CURVE>. |
||
| − | |||
| − | rt <SERVICE-CURVE> |
||
| − | <SERVICE-CURVE> sets the amount of bandwidth a leaf class is |
||
| − | guaranteed. rt is ignored for inner classes. This amount of |
||
| − | bandwidth is always available to it regardless of the other |
||
| − | classes use of the link or the maximum bandwidth allowed by |
||
| − | this class or its parents. When calculating <SERVICE-CURVE>, |
||
| − | the calculation of the start time of <BURST-BPS> is complex. |
||
| − | Roughly, it is the last time the class was backlogged and was |
||
| − | under the guaranteed rate. Every leaf class must have a ls or |
||
| − | rt <SERVICE-CURVE> specified. |
||
| − | |||
| − | ul <SERVICE-CURVE> |
||
| − | For classes with a ls curve, set the maximum bandwidth this |
||
| − | class and its children are permitted to use. This only effects |
||
| − | sending packets under the ls <SERVICE-CURVE>, but packets set |
||
| − | under the rt curve are included in the bandwidth calculation. |
||
| − | When calculating <SERVICE-CURVE>, <BURST-SEC> is measured from |
||
| − | the last time this class was backlogged, ie had packets waiting |
||
| − | to be transmitted. |
||
| − | |||
| − | <SERVICE-CURVE> |
||
| − | --------------- |
||
| − | |||
| − | A service curve specifies how much bandwidth a class can use. It |
||
| − | has an optional initial burst rate, followed by a steady state. |
||
| − | The start time of the burst depends on the service type (rt, ul or |
||
| − | ls). The are two syntaxes available for describing a service |
||
| − | curve. In both you use the usual tc units to designate speed, |
||
| − | data size and time. |
||
| − | |||
| − | [m1 <BURST-SPEED> d <BURST-SEC>] m2 <STEADY-SPEED> |
||
| − | <BURST-SPEED> is the speed of the burst, and <BURST-SEC> is the |
||
| − | length. Thereafter the class sends at <STEADY-SPEED>. |
||
| − | |||
| − | [umax <BURST-BYTES> dmax <BURST-DSEC>] rate <STEADY-SPEED> |
||
| − | "rate" has an identical meaning to "m2" above, so it specifies |
||
| − | the long term speed the class is entitled to. To determine what |
||
| − | "umax ... dmax ..." means, an implied burst speed is calculated |
||
| − | using "<BURST-BYTES> / <BURST-DSEC>". If this burst speed larger |
||
| − | than <STEADY-SPEED> then this syntax is equivalent to "m1 |
||
| − | (<BURST-BYTES> / <BURST-DESC>) d <BURST-DSEC> m2 <STEADY-SPEED>" |
||
| − | where the stuff in (...) is calculated. If the implied burst |
||
| − | speed is not greater than <STEADY-SPEED> then during the burst |
||
| − | period effective specification becomes "m1 0 d (<BURST-DSEC> - |
||
| − | <BURST-BYTES> / <STEADY-SPEED>) m2 <STEADY-SPEED>". This means |
||
| − | the class is not allowed to send any data at all during the |
||
| − | burst period. The burst period is the time left over after |
||
| − | if <BURST-BYTES> were sent at <STEADY-SPEED>. |
||
| − | |||
| − | |||
| − | Classes. Traffic is assigned to HFSC classes using a filter. |
||
| − | |||
| − | Scheduling. The HFSC scheduler sends packets required to meet |
||
| − | guaranteed bandwidth specified by rt, but only if those requirements |
||
| − | would be broken by not sending them. In that case it sends the |
||
| − | packet that will complete sending closest to its deadline. If not |
||
| − | forced to send packets required by rt it will send a packet according |
||
| − | to the ls criteria, choosing the packet that will complete sending |
||
| − | closest to its deadline. For both rt and ls, time is measured with a |
||
| − | clock running at one tick per bit per second of bandwidth allocated |
||
| − | by the <SERVICE-CURVE>. Thus specifications with higher bandwidth |
||
| − | appear to have their deadlines arrive sooner, and packets appear to |
||
| − | be waiting longer and thus are sent sooner in actual time. |
||
| − | |||
| − | Policing. HFSC discard packets that aren't classified unless |
||
| − | the default classid is specified to the qdisc. |
||
| − | |||
| − | Rate Limiting. HFSC restricts classes to the rate limit imposed by |
||
| − | the ul specification, and the rate limit imposed by its ancestor |
||
| − | classes on their children. Classes that have exceeded their |
||
| − | real time <SERVICE-CURVE> in the past may be rate limited during if |
||
| − | they don't also have a link share and the link is backlogged. |
||
| − | |||
| − | Classifier. The HFSC queuing discipline does not classify packets. |
||
| − | </PRE> |
||
