Pseudo Language

I'm currently converting a substantial application to be Internationalized and Localized. This basically means we're completely abstracting the language from the application, so we can easily have French, German, Italian, and other languages displayed in our application.

In addition to internationalization, we also do localization, which means the way we format dates, times, currency and other "things" are specific to a given locale. For example, there are French speaking people in both Canada and France, but the way they represent numbers, dates, times, and currency can drastically vary, so it's important to have formatting specific to both locales.

To help us with this, we have a bunch of cool, nifty tools. One of the tools we have is some code that generates a pseudo language. The pseudo language is basically a fake language that reads like English, but looks obviously different. The pseudo strings end up looking sort of like the original strings, but with heavily modified characters, and additional padding on the beginning and end of the strings. For example, the string:

The quick brown fox jumps over the lazy dog.

...becomes:

[!!! Ťħĕ qǔĭčĸ þřŏŵʼn ƒŏχ ĵǔmpš ŏvĕř ťħĕ ľäžЎ đŏğ. !!!]

The idea behind the additional length (which, by the way, is variable depending upon the length of the original string) is to ensure there's enough room on the forms in case the translation ends up being substantially longer than the original. The idea behind the modified language is to have some obvious way of determining that our application isn't loading English strings (which, for reasons that shall remain unexplained, can be sort of a pain in the butt on .NET). It's also nice because it gives a visible indicator of places that aren't being translated.

One issue we quickly found, however, was that our pseudo translator was rather stupid. The following string, which contains .NET String.Format arguments:

This is a string with String.Format parameters: {0:D}

...would end up looking something like:

[!!! Ťħĭš ĭš ä šťřĭʼnğ ŵĭťħ Šťřĭʼnğ.Fŏřmäť päřämĕťĕřš: {0:Đ} !!!]

The issue with this, of course, is that {0:Đ} doesn't mean anything to String.Format--{0:D} means something to String.Format. So, we had to modify the code to be aware of {} blocks and not attempt to translate the contents. In theory, this sounded straight forward, but it ended up being a bit involved. One thing is that "{{" and "}}" is how String.Format escapes the "{" and "}" characters, so a perfectly valid String.Format statement is:

This {{{0}}} is a {{ totally valid {{ String.Format statement.

...or maybe:

This is yet another perfectly valid String.Format argument: {{{0:D}{{Hello World!}}

...in the first, String.Format produces "This {0} is a { totally valid { String.Format statement." (the "0" can end up being any number of things, but the point is it's going to be wrapped in {} in the output) The second ends up looking like "This is yet another perfectly valid String.Format argument:{0:D{Hello World!}" So for inner {}, String.Format interprets those as places for format arguments. Any outside {{ or }} get interpreted as braces in the resulting string.

The solution I used kept track of the previous character. If I encountered {{ or }}, I reset the previous character. Any single instance of { encountered resulted in no translation until a } was encountered.

Leaking Memory with Marshal.AllocHGlobal

The other day I was going through the source code in our application and tidying up various internationalization tasks, and I came across the following code:

    IntPtr BinaryValue = Marshal.AllocHGlobal(8 * Marshal.SizeOf(new byte()));

I quickly noticed that this call didn't have the requisite Marshal.FreeHGlobal call, which basically meant it would leak 8 bytes every time this code was executed. Going through the application, I then found ~15 other places, each of which would leak anywhere from a few bytes up to a couple hundred.

Leaking memory in .NET? Say it isn't so! According to the documentation, the Marshal class:

Provides a collection of methods for allocating unmanaged memory, copying unmanaged memory blocks, and converting managed to unmanaged types, as well as other miscellaneous methods used when interacting with unmanaged code.

So one important thing to keep in mind with the Marshal class is that it was specifically crafted to allow .NET to interact with unmanaged memory, among other things. This means particular care must be exercised, because many of the methods have requisite cleanup steps, or deal directly with memory. Admittedly this can be difficult to cope with, especially after programming in a language where memory management is typically handled entirely under the covers.

Anyways, after finding and establishing that the above code was leaking memory, the question quickly moved on to: what is the best way to make sure memory is always released? Anybody who matriculated to the .NET world from C++ is acutely aware of the perils of non-deterministic finalization, so wrapping it in a class and relying on a destructor is out of the question. One quick/dirty solution a co-worker of mine proposed was a try/catch/finally block:

IntPtr BinaryValue = IntPtr.Zero;
try
{
    BinaryValue= Marshal.AllocHGlobal(8 * Marshal.SizeOf(new byte()));
}
catch(Exception)
{
}
finally
{
    if (BinaryValue != IntPtr.Zero)
    {
        Marshal.FreeHGlobal(BinaryValue);
    }
}

...the benefit of this is, the finally block will execute regardless of how this code exits. Also, the catch block can be omitted and the finally block will still execute, so this code is also nice for places where it'd be better for the caller to handle the exception.

Another option is the using statement (not to be confused with the using directive). The using statement is actually one way to make .NET behave in a more deterministic manner; once the using block goes out of scope, objects declared in the using block will be zapped from memory. The only requirement is that the object declared in the using statement has to implement IDisposable, so this means we have to wrap Marshal.AllocHGlobal in a wrapper class. I wrote a quick and dirty wrapper class:

class UnmanagedMemory : IDisposable
{
    private IntPtr _ptrToUnmanagedMemory = IntPtr.Zero;

    public UnmanagedMemory(int amountToAllocate )
    {
        _ptrToUnmanagedMemory = Marshal.AllocHGlobal(amountToAllocate);
    }

    public IntPtr PtrToUnmanagedMemory
    {
        get { return _ptrToUnmanagedMemory; }
    }

    public void Dispose()
    {
        if (_ptrToUnmanagedMemory != IntPtr.Zero)
        {
            Marshal.FreeHGlobal(_ptrToUnmanagedMemory);
            _ptrToUnmanagedMemory = IntPtr.Zero;
        }
    }
}

...pretty straight forward. Using this class then looks like:

using (UnmanagedMemory um = new UnmanagedMemory(8 * Marshal.SizeOf(new byte())))
{
    // Do something here with um.PtrToUnmanagedMemory
}

...if you set a break point right after the using block exits, and one in the Dispose method of UnmanagedMemory, you can see the "deterministic" finalization in action. This method is a little more involved, but probably preferable; the UnmanagedMemory class could be expounded upon to provide any number of useful features, and it's cleaner and more object oriented than the try/catch/finally block.

(worth noting is that the using statement is essentially a try/catch/finally block, under the covers, but it is still fundamentally different to wrap unmanaged memory in a class and interface with it like that. I believe this method is preferable)

In our application, we ended up using the try/catch/finally block, mostly because of circumstances inside the company (release coming up, can't do anything too wild) and because I needed tight control over the exception flow so I didn't have to fundamentally alter any of our error handling code. Eventually, though, I'll head over to using something like UnmanagedMemory.

Population of Rome, Continued

Another question often pondered when people discuss the population of ancient cities is: what was the average life expectancy? The formula for computing the average expectation of life is pretty simple:

AverageExpectationOfLife = SumOfTotalYearsLived / TotalIndividualsRecorded

The above formula is basic enough, but Tim G. Parkin, author of Demography and Roman Society, points out the rub:

The simple equation...may often produce what appears to be a realistic figure for average life expectancy, but when ages at death are sorted into age groups (say, 0-1 years, 1-9, 10-19, etc.), serious problems become evident. That infant mortality (i.e., deaths in the first year of life) is grossly underrepresented in the tombstone records has long been recognized; clearly infant deaths, even when burial took place, went largely unrecorded, or at least any such records have not survived in large numbers.

...in other words, infant mortality skews the number so heavily that one thinks of the average lifespan as being low, but people surviving past childhood have a good shot at living much more reasonable (by modern standards, anyway) lifespans. This book also contained some eye opening information on Roman demographics, and it seems the same uncertainty surrounding the estimation of water abounds.

Another interesting point discussed in the book is the ratio between sexes; Beloch and previous authors (Carcopino's Daily Life in Ancient Rome: The People and the City at the Height of the Empire., for example) assumed an equal sex ratio. The evidence put forth by Parkin suggests otherwise, although he makes it quite clear that even his evidence may be off:

Another very striking feature when considering the thousands of epitaphs collected is the preponderance of males over females. If we assume for the moment that in reality the sex ratio over all age groups was approximately equal, this means that in our sample three males were commemorated for every two females...It is true that there is some other evidence...that suggests that males really did outnumber females in certain periods of antiquity.

Another question: how many children did Roman families have? And were the Romans generally monogamous, or did males sometime take multiple wives? The latter question was answered with relative ease: the Romans were monogamous, and generally frowned on polygamy. The former is perilous. Given an average life expectancy of 25 years, a gross reproductive rate of over 2.5 is required to require a stationary population. In other words, each woman needs to bear 2.5 daughters, or about five children. More than that would be population growth; less would be decline. That said, Parkin brings up many issues with estimating Roman family size:

• It was known how may children were born (or often only sons), but not how many survived to adulthood.
  
• Large families may be mentioned because they were abnormal, skewing modern perceptions.
• Daughters often remained anonymous or unmentioned.
• Ancient authors rarely show any demographic interest in average family size.

Through Roman Egyption census records, the average size of households was calculated to around six, including parents and slaves.

Based on the information Parkin brings up, it seems like it should be possible to refine Beloch's

The Population Of Rome

Determining the population of Rome is a rather tricky issue. It's been much debated, and the estimates vary substantially depending upon who's numbers you look at. The research also tends to focus on a relatively narrow window of Roman history (46 BC to AD 15), and tends to focus on a subset of the overall population: recipients of the corn dole. Beloch's 1886 paper, Die Bevölkerung der griechisch-römischen Welt, is the starting point of discussion surrounding the corn dole.

The corn dole (sometimes referred to as the grain dole) was a handout by the Roman government to citizens of Rome. The number of people who received the dole varied, but research shows it varying between 150,000 to 300,000+. The dole started in 123 BC by Gaius Gracchus, and in 58 BC Publius Clodius Pulcher made it completely free. Beloch used the number of qualified male citizens who received the dole as a starting point for estimating the population of Rome. Based on this, he then estimated the number of dependent women, children, and slaves. He also estimated the number of foreigners in the city as well. This resulted in a figure of ~800,000 people inhabiting Rome.

As a verification of the above estimate, Beloch estimated the grain consumption of his computed population and compared that with estimates of the total amount of grain flowing into Rome each year. Here is a table from Gerda De Kleijn's book, The Water Supply Of Ancient Rome - City Area, Water, and Population, which shows the years of the dole and number of recipients:

TODO scan table

There are several issues with this, and especially so when attempting to reconcile the figures with Rome's water supply. For starters, the narrow window of data points is particularly troubling: Beloch's data spans 60 years, but aqueduct information spans in excess of five hundred. Superimposed on our previous graphic of Rome's water supply, it's easy to get a sense of just how small the window really is:

TODO UPDATED GRAPHIC




Another question often pondered when people discuss the population of ancient cities is what the average expectation of life was. The formula for computing the average expectation of life is pretty simple:

AverageExpectationOfLife = SumOfTotalYearsLived / TotalIndividualsRecorded

Here is a graph showing how much the water capacity in Rome increased over time. The y-axis is in quinariae, and the x-axis is time:



...the above numbers were computed using Blackman and Hodge's figures. Notice that this graph is similar to the one I previous posted that tabulated totals in Excel. The big difference with this graph is the x-axis is appropriately scaled with respect to the aqueduct's construction date.

In the perfect world, I'd have enough data to correctly taper each of the horizontal lines, clip out sections where the aqueduct was incapacitated or otherwise non-functional, etc. But it isn't a perfect world, so I have to settle for the above. (which for all intents and purposes isn't that bad)

I recently reviewed another paper by H. Chanson, Hydraulics of Roman Aqueducts:
Steep Chutes, Cascades, and Dropshafts, which had a totally different approach to estimating water capacity. Chanson actually evaluated archaeological evidence to determine the hydraulic flow in ancient Roman aqueducts. His findings are quite interesting: "maximum" flow for the nine major aqueducts was 1,013,960 cubic meters per day. Adding in the Traiana and Alexandrina (I guess there's enough evidence to find their hydraulic potential), Rome was supplied with 1,148,960 cubic meters, per day. The graph for the data tabulated in Chanson's paper looks like so (in this graph, the y-axis is cubic meters per day):



Chanson's data also makes me think that Grimal's data is probably based on hydraulic potential rather than Frontinus' records. And this is probably fine. One interesting thing about these graphs is regardless of the magnitude of the units, they look pretty similar. As in, the overall progression of water flow into Rome is pretty obvious; the overall magnitude seems to be of contention, but not the fact that it A) increased, and B) increased by some proportional amount.

One important thing to note with Chanson's data is that this is the maximum capacity, so it gives us a well-defined upper bound for the aqueduct's delivery potential. However, I doubt the aqueducts were ever running at capacity, and it's probably safe to say that it was common for them to run at half-capacity (or less) during dry months (being spring fed means the water source wasn't constant). So the estimates around ~600,000 cubic meters per day still seem more reasonable to me.

I've also recently been reviewing sources on population; I'll be posting more about that very soon.

I decided to look a bit more into the water quantity figures Hodge listed in his 2002 work, Roman Aqueducts and Water Supply. It turns out that his figures are derived from the work of a previous author, Grimal. I know little about Grimal and some google searching didn't yield much information beyond basic facts: he's French, and most of his publications seems to have occurred in the 60s and 70s.

Purportedly, his figures are based on the records of Frontinus, but I don't possibly see how that could be: Grimal's cumulative total of the nine aqueducts for which Frontinus supplied measurements is fully double the total of other authors, including Hodge's figures in his collaborative work with Blackman. Almost all estimates are based on the figured supplied by Frontinus, and tend to be in the range of 12,000 to 15,000 quinariae. Grimal is hovering around 25,000.

The generally accepted conversion from a quinaria to some unit of volume over time appears to be about 40 m³/day. As always, estimates vary; some seem to have a more conservative figure of 20 m³/day, and I saw a few estimates above 40 as well. To get a sense of what 40 cubic meters of water actually looks like, think of it as being a cube of water ~3.41 meters on all sides, or about ~10,000 gallons of water. A swimming pool 24 feet round and four feet deep has about 12,000 gallons in it.

Knowing this, Grimal's total for the water supply of Rome is 24,805 quinaria per day, which Hodge seems to happily accept and convert to cubic meters using the 40 m³ estimate: 992,200 m³ per day. To get a sense of how much water this is, this would be a cube of water 99.74 meters on all three sides. This is 262 million gallons of water. Per day. Assuming Rome contained a million inhabitants (which seems to be the generally accepted estimate), this means that the water supplied 262 gallons of water for each person in Rome, per day.

Personally, I find Grimal's figures wildly suspect. For starters, Grimal arrives at totals double of other estimates. An "alternative estimate," according to Hodge, was H. Fahlbusch's proposed range of 520,000 to 635,000 m³. Other estimates seem to fall in this range as well.

Second, looking at what typical US cities used, it seems really unlikely to me. In a paper by Morgan, he gives the following table for U.S. cities in 1900:

...assuming the above figures are correct (they seem reasonable, but I haven't verified them myself), Hodge/Grimal are stating that Rome supplied more water per person than most major cities in the U.S. did by the start of the 20th century. I find that a very hard pill to swallow. I'm not alone on this: in Morgan's paper, his estimates are based on Frontinus' figures, and he arrives at a more typical ~14,000 quinaria, which using the 40 m³ conversion would end up being 560,000 m³ of water per day. However, Morgan settles on a more conservative figure of 6000 gallons/day per quinaria (22.7 m³/day), which makes his estimate 318,000 m³ per day. This is fully one-third of Hodge/Grimal's estimates.

Worth note is that when the Aqua Traiana and Aqua Alexandrina are added in (which Grimal estimates at 113,920 m³ and 21,160 m³ per day, respectively, although one what basis these estimates were made is unknown), it pushes the Rome total to 1.127 million m³ per day. Considering most other authors don't even bother supplying figures for the Traiana and Alexandrina, I wonder A) where Grimal got this information, and B) on the basis of what estimations. It certainly wasn't Frontinus' estimates.

What do modern aqueducts supply? The Catskill Aqueduct, which begins at the Ashokan Reservoir in Olivebridge, New York, in Ulster County and runs 120 miles to New York City, has an operational capacity of about 580 million gallons (219,240 m³) per day. It typically carries less than this, with flows averaging around 350 - 400 million gallons (132,200 - 151, 200 m³) per day.

Another example--the Colorado River Aqueduct--runs 242 miles from the Arizona-California border. Its capacity is 1.3 million acre-feet (1.6 km³) per year, or 1.6 billion cubic meters per year, or ~4.4 million cubic meters per day. Average flow is probably less than this, but it's a staggering number to think about. The construction project lasted eight years, and employed a total of 30,000 people.

I won't even list the figures for the California Aqueduct. It'd make your head explode. :)

Perhaps the most appropriate comparison, however, is with another Roman Aqueduct: Pont Du Gard. The water conduit, which is 1.8 meters (6 feet) high and 1.2 meters (4 feet) wide and has an average gradient of 0.4 percent, supplied the Roman city of Nemausus (now Nîmes) with an estimated 20,000 cubic meters of water per day (note: this is an uncited figure from wikipedia; I'll have to look into how that number was arrived at). This is also 500 quinaria, using the typical 40 cubic meter conversion rate.

Using the cube analogy, Pont Du Gard supplied Nemausus with a daily 27.1 meter cube, or 5.3 million gallons a day. Estimates for the population of Nemausus were around 50,000 inhabitants, so that would be roughly 106 gallons per person, per day, which is a pretty far cry from Hodge/Grimal's estimate of 262 gallons person/day in Rome.

Aqueducts

I'm currently taking a class on Roman aqueducts. We discuss and research the aqueducts that supplied Rome (and other portions of the Roman empire) from a number of different perspectives, including historical, hydrological, cultural, economic, logistical, etc. Basically, it's a class covering a broad range of topics pertaining to Rome's water infrastructure, with the aqueducts at the core of discussion.

As part of the class, we have to do research on a topic of our choice. Originally, I wanted to do research on how the aqueducts reduced the instance of water-borne disease, but my preliminary research revealed a rudely obvious fact: getting data on that subject was going to be close to impossible. I'd also made some crude assumptions, such as assuming that water-borne diseases like Cholera that produced epidemics during the 19th century were present (they were not--Cholera was endemic to the Indian subcontinent until the 18th and 19th centuries). Basically, it was doomed from the get-go.

With that fresh defeat behind me, I got to thinking--how did the water supply effect the population of Rome? How did the population of Rome effect the water supply? The aqueducts were built over a span of 500 years, so one thing I plan on examining is how the water supply grew in relationship to the population growth of Rome during 312 B.C. and A.D. 200. I'd like to attempt to answer whether or not new capacity was added in response to demand, and if the added water in turn sparked increased population growth.

Admittedly, this task is also fraught with peril: gigantic discrepancies in the capacity of the aqueducts exist in current research. Archaeological evidence and historical accounts of their capacity are obviously suspect, because stuff generally isn't built to last 2000+ years (even the Romans, notorious for their over-engineering, aren't up to snuff), and historical records are sparse and sometimes cryptic.

Another huge stumbling block is the actual population of Rome--again, estimates vary greatly, from a peak population of 200,000 inhabitants (absurdly low) to 2,000,000+ (absurdly high). The general consensus among reading seems to be around a million, but it's an exceptionally wide range. Also unknown is the rate of growth, epidemics that may have caused lapses in growth, immigration, the effects of being sacked seven or eight times, famines, etc. The demographic information leaves much to be desired.

Still, the estimates are good enough to attempt to draw some conclusions. And the idea sounds entertaining to me, so...without further ado:

...the above is a chronological listing of Rome's aqueducts. I have the dates as negative numbers because I wanted to do some basic graphing in Excel. Anyways, this chart shows their rough completion date, overall length, altitude of water source, altitude of the water level in Rome, and the total portion of the aqueduct that was above ground. Much of this information is from the meticulous records of a model public servant by the name of Sextus Julius Frontinus, from which most of the "hard" data today is derived. Because Frontinus died before the completion of the Aqua Traiana and the Aqua Alexandrina, little is known about those aqueducts other than some scant archaeological evidence.

Here is another chart, showing the "output" (in quotes for a damn good reason--more on this in a bit) of each aqueduct in quinariae, according to several sources:

Overall B&H is "Blackman and Hodge," who had estimates based around modified figures derived from Frontinus' records. Hodge later tabulated greatly increased values in a subsequent publication, hence the separate column. The last column are figures from Gerda de Kleijn, who also made estimates based on modified figures from Frontinus.

The first thing to notice about this table is the vast discrepancy between the last four columns. All four columns represent estimates by published sources. I'm going to do more research into this discrepancy, but for now, I'll let it be. The second thing to notice is that these units are all in quinariae.

Much of the controversy over the total water "output" supplied by the aqueducts stems around the rather nebulous nature of a quinaria. A quinaria was a Roman unit of area, roughly equal to 4.2 square centimeters. Its primary use was to measure the cross-sectional area of pipes in Roman water distribution systems. Even in Roman times, there was controversy over the value of a quinaria--according to Frontinus:

Those who refer (the quinaria) to Vitruvius and the plumbers, declare that it was so named from the fact that a flat sheet of lead 5 digits wide, made up into a round pipe, forms this ajutage. But this is indefinite, because the plate, when made up into a round shape, will be extended on the exterior surface and contracted on the interior surface. The most probable explanation is that the quinaria received its name from having a diameter of 5/4 of a digit...

...in other words, Frontinus disagreed with Vitruvius about the actual value of a quinaria (primarily on the basis of Vitruvius' estimate being "imprecise" with respect to inner and outer circumference). Regardless of the Romans haggling over the value of their own units, one thing is certain: using quinariae to measure flow would be somewhat like using feet to measure acres, or kilograms to express newtons. The Romans did have units of volume, but the issue with the water flow is that it's a measurement of volume over time, which the Romans had no accurate or practical means of measuring. Culleus per elapsed sundial just doesn't roll of the tongue quite like cubic meters per second. Considering they had no accurate temporal measuring devices, it's debatable if the Romans even grasped the concept of units expressed over time.

To get a better sense of how the flow of water changed over time, I did some graphing in Excel:

The x-axis is time, and the y-axis is cumulative water flow in quinariae. The different colors correspond with a specific aqueduct. This is my first stab at graphing some of these data, so I plan on doing much more here; I want the temporal axis to be to scale, rather than the regular intervals in the above graph, but Excel seems suited only for the most basic charts. I suspect I'm going to have to draw much of these by hand.

Here's another interesting graph that shows how dependent Rome was on each aqueduct over time:


...considering the scale of aqueduct construction, new projects were not to be taken lightly. So there must have been a practical reason to add new capacity--could it be population? Who knows. :)