Batteries not included: what should go in the C++ standard library?

About forty Christmases ago I was absolutely delighted to open a slot car racing set from my parents. I feverishly set everything up, created a track reminiscent of Brands Hatch, and went to plug everything in, only to discover that I needed batteries to operate the controllers. This was Great Britain in the 1970s. There were no shops open on Christmas Day. There were no appropriate batteries in the house, nor in our neighbour’s house. My delight was transformed to desolation. My parents had an argument. My grandfather distracted me with a story about one of his second world war sorties over the French coast. I watched a James Bond film sulkily. All because there were no batteries in the box.

What prompted this blog post?

With the next C++ Standard meeting in Jacksonville bearing down on us, I have just submitted the next version of the 2D graphics paper, One argument levelled against it runs along the following lines:

“A graphics library shouldn’t be part of the standard library. The standard library is too big already, and this will make it so much bigger. It’s just not appropriate. The library should only contain small types.”

Of course, I disagree with this. I think the library is impoverished and needs fleshing out to make C++ a stronger contender in the development arena.

What is already there?

A quick glance at shows the complete set of headers divided up into the following categories:

  • Utilities
  • Strings
  • Containers
  • Algorithms
  • Iterators
  • Numerics
  • Input/output
  • Localisation
  • Regular expressions
  • Atomic operations
  • Thread support
  • Filesystem

Some of these are more useful than others, and some attract more irritation than others. I know some developers who complain about the input/output, saying it’s bloated and inefficient. I know others who bemoan the incompleteness of the thread support. I don’t know anyone who actually uses the regular expressions, nor the localisation, nor std::multimap.

Then again, who doesn’t use std::vector, std::array, std::iterator, std::find_if and so on? There is plenty to choose from, and some parts find greater application than others.

What could go in the standard library?

One argument is that the standard library should be minimal and complete. It should contain types that require compiler support, like the type traits, and things that are strictly platform specific, such as SIMD support, file IO and atomic operations. The rest is just holding the user’s hand. I’m going to call this level 0 for want of a better description.

If you asked a user to write a mutex, I’m pretty sure they would get it wrong several times before they succeeded. Most likely they would not actually get it completely right and would encounter a peculiar edge case that broke everything at an inopportune moment. It suggests another class of content: things that are hard and error prone. This obviously includes thread support and also memory management. I’ll call this level 1.

Then there are the obvious types that it would be silly to do without. These are the types that are frequently reinvented and are the basic staples of many programs. These are the types such as std::complex, std::array, std::function, std::string, numerics and so on. They aren’t necessarily hard to write, but they are extremely broad in application and it is a safe bet that they will be used by nearly every programmer who will be grateful to have them. This is level 2, and standard commentators will start to murmur a little, remarking on how std::string is an unnecessarily large interface that should be made up of non-member functions, and std::function shouldn’t perform allocations.

What’s left on the list? The STL is one of the wonders of the modern age as far as I’m concerned but it doesn’t fit into any of the prior categories. The containers, iterators and algorithms aren’t “obvious” types, and there’s no particular need to have them in the library, but they’re incredibly useful. This is level 3, and here is where standard commentators start to get quite restless. They might remark that these types and functions are really quite large and bulky, they soak up implementer time, and they should have remained in a separate library.

What do other languages have?

Hand on heart, I’ve been full time C++ for twenty years now, with very little exposure to other languages, so I can’t speak from much experience. However, when you look at Java and C#, you find absolutely huge library support “out of the box”.

The Java Class Library consists of, at first glance, support for UI, database connection, networking, sound, image manipulation, XML, CORBA, encryption, even hosting for scripting languages: the list goes on.

If you decide to open Visual Studio and start writing a program using C# you have the Base Class Library (BCL) at your disposal along with the .NET Frameworks (FX) which together include support for networking, reflection, threading, XML, and parallelism amongst others. There is tooling for building GUIs too, and of course there is LINQ.

These languages are made a little differently from C++ though. Java is owned by Oracle, C# is owned by Microsoft, but C++ is owned by no corporate entity. The development of C++ is controlled by a public process which makes it rather slower to grow. Meetings to discuss, and vote on, what should be added and amended must be inclusive and public. However, there is an enormous wealth of support from these languages that would fit in at level 3, and it seems obvious to me that this support contributes to the popularity of those languages.

Having said this, it’s important to note that bigger is not necessarily better. I don’t think it is controversial to remark that these frameworks err on the large side. However, there is a useful core which I contend is larger than the standard library. That useful core drives the success of these languages. There is clearly a “critical mass” that can be achieved, and while those languages have erred on the other side of that mass, C++ has yet to reach it. I would rather we overshot a little and reaped the benefit of greatly increased adoption.

What typical jobs can’t be done “out of the box”?

Consider my opening anecdote of a Christmas in 1970s Great Britain. A more adult version of this is experienced by many learners who try and solve programming tasks using C++. Would you like to open a socket? Nope, can’t do that. Decompress a stream of data? Sorry, no. Open a dialog box? No. How about telling me what pixel my mouse is pointing at? Ummm… no. Flash some LEDs? Play a sound? Turn off the power? No, no and no.

It’s still the 1970s in C++ land.

What do we do instead?

Your toolchain will come with bindings for interfacing with the host operating system, which should solve network, power, sound and UI issues.

There is also a huge ecosystem of libraries. It really is impressive looking at the amount of code out there for people to choose from to solve your problems. Boost contains an Aladdin’s cave of riches. GitHub is overflowing with niche solutions for all manner of problems.

However, this tyranny of choice is paralysing for newcomers to the language. Operating system bindings differ for each OS. A fresh user cannot sit down with one book and any toolchain and learn the language through solving some real world problems: there is other, platform specific stuff to know.

Not only that, but not all code shops will permit the use of third party code beyond a language and its library. There are commercial, industrial and military environments that insist all code is created in a “clean room” environment. The arrival of software patents has made this an even more complicated matter.

Is there another vehicle for larger types?

As I said, I have skin in this game. I think a standard 2D API will be a great benefit to the library and I have one that’s nearly ready to go. I am keen to turn it into a Technical Specification and then see it incorporated into the International Standard.

But is there a destination other than the IS where it might be published?

At the moment, no, there isn’t, but I can envisage a repository of source libraries that have been “blessed” by the committee which exist outside of the standard. At the moment the committee consists of two streams, the library and the language. A third stream could be introduced which reviews candidate libraries for inclusion in this repository, rather than being added as level 3 contributions to the International Standard.

This repository could come with clear rules:

  • All conforming implementations would have to be able to build it entirely.
  • Contributions should not be burdened by licensing restrictions.
  • Updates to the source should be reviewed by the committee.

What I’m talking about is a package index, or better still, a package manager. The former will retrieve libraries, while the latter will manage dependencies and auto-updates. If you’re writing some code in a blog post that uses graphics, you should be able to instruct your readers with something like

To build this code, type "cpp-get lib-graphics 1.56" at the command line.

and it should Just Work. It means we can assume every developer has access to a particular library.

There are already several developers looking at ways to implement a package manager. There are some mature efforts out there. Conan, apt-get  and vcpkg are a godsend: indeed, the 2D graphics paper’s reference implementation relies on vcpkg to install the support libraries. But, and it’s a big but, we don’t have a standardised package manager or index, like Python’s PyPI (over 120,000 packages!), ready to power any C++ implementation with the extra voltage needed to write truly useful programs out of the box.

In summary, perhaps there are two ways to ship libraries that can be assumed to be accessible to all C++ developers:

  • In the box (i.e. in the International Standard)
  • In a package index (To Be Designed)

I hope that at some point in the near future something will emerge blinking into the light of day and we can examine an additional way of growing the standard library.


This post started as a pair of email conversations. One was with Titus Winters, chair of the Library Evolution Working Group. Several portions are lifted with gratitude directly from his words. The other was with Herb Sutter, chairman of the ISO C++ Working Group, and Richard Smith, C++ project editor, to whom the same thanks are directed. Many thanks also to Herb for review commentary and corrections on the organisation of C#. Remaining errors are of course my own.

17 thoughts on “Batteries not included: what should go in the C++ standard library?”

  1. Hi Guy,

    first of all thank you for this very interesting blog post.
    And thanks for going through all the troubles to contribute to the C++ Standard.

    Having said that I have to admit that I am actually not quite sure how much more of a standard library is required. I think there are some nice-to-have parts like the plf::colony class or a std::inplace_function and better support for asynchronous, optionally multi-threading, event-driven programming so I definitely think that some additions could be made.

    I am pretty reluctant with some Boost-inspired additions, filesystem being kind of in between but I think just adding ASIO is not doing the language a big favor. Maybe people should think about asynchronous programming models like job queues and job stealing (either by explicit consumption or worker threads). But one has to keep in mind that most of those solutions will be very general purpose and that might actually be the problem with them because they may not be general purpose enough to be thrown at very particular problems that need to be solved. Maybe that can be achieved by super generalized code that goes through lots of templated policy types or something similar but then again usefulness might be limited due to high configuration overhead.
    Anyway, something like this should be used as the core for some asynchronous I/O library instead of throwing all the parts from ASIO with their service instances and strands and whatnot in some std::aio namespace (or wherever those gnarly beasts will reside in the end).

    So the real question is whether all those (often very useful) things are well-suited enough to be sitting at the core of the language. I would like to take this opportunity to make a different point by taking a step back and looking at the larger picture.

    You mentioned some kind of “package index” and “package manager”. I know there are lots of solutions for this but none of them are quite “standard-ready” (as you pointed out). On my Linux machine I have Portage, a highly configurable build-from-source manager that has lots of advantages. One disadvantage being that it works on system-level and often you don’t want to update your system libraries (e.g. because you might break dependencies by requesting incompatible versions, etc.).
    And of course it’s highly platform-specific but that could probably be remedied. But I am not advocating this, I am just taking it as an example.

    So anyway: while a global package repository is a good thing to have it is even better to add the option to build and install packages on a per-project basis – locally, so to speak. This in turn is heavily coupled with the build system that is used (my current favorite being CMake but just for lack of a better alternative). And this is where I am actually heading.

    Now, I think from what C++ really suffers is this “we don’t care how you compile or how you are going to be able to use modules” attitude. I know it’s probably hard to push development into a more infrastructure-like direction but that could be worthwhile. Especially considering the tremendous time I spend looking at the C++ compiler to parse a million header files and templates, re-instantiating them for every single translation unit just so the linker can throw gigabytes worth of object data away again. This makes me weep at all the trees that get burned while my CPU is drawing a lot of power and heating up the room. Not to mention the inability to do source-level transformations (apart from “limited” templates and some constexpr magic) or do other code-injection trickery like instrumentation for performance monitoring. And don’t get me started on reflection (which will hopefully come soon). So maybe it is time for some standard, platform-independent IR (kind of like Khronos SPIR-V) that can be parsed and understood and transformed by tools so that we can finally stop throwing millions of lines of code at the compiler for each C++ TU.

    So to sum this rather long reply up:
    A package manager would allow us to keep the core clean and add-in advanced functionality as we need it, leaving us with the freedom to choose which technology to choose (given that there are several packages doing roughly the same for us)
    Focusing more on the infrastructure part could open up lots of new possibilities, massively reduce compile times and let us iterate faster on code.

    Maybe all this requires another standard effort (as it’s not actually C++ per-se) but I would definitely like to hear your thoughts about that.

    Maybe this was not the reply you hoped for but I had to get that out 🙂

    May the force be with you. Kind regards

    Dennis Philipps

  2. Oi! Multimap is excellent! It provides a container where you can have multiple items attached the same key and it’s automatically sorted by key as you go – which is the big selling point.

    I once used it to take a job that used 400 machines and 6 hours to one that took 30 minutes and a few dozens machines with a fairly small refactoring of the code. Don’t knock it!!

    1. I’m pretty sure Guy knows how a multimap works, the main bugbear people have with it in general – much like list and unordered_map – is that it’s design prevents cache-friendly implementation, in most cases – however depending on your application this may vary, and in your case obviously it helped a lot, which is great.

  3. Package managers are convenient. However they tend to introduce problems.
    They don’t resolve dependency hell. They don’t often handle version conflicts well. They tend to have zero real vetting except for corporately backer packages. This results in hordes of specious quality packages, abandoned packages and purely junk packages that persist. They tend to induce a false sense of security around third party code.

  4. I am all for the second option – a library outside of the standard, using std programming idioms, with no dependencies besides the std library (which in itself should be minimal). I would applaud any such open-source effort.

    Trying to add non-essential things into the standard could easily lead into a feature creep trap: “while we add this it would be only logical to add also this and this…” Before you know it, you’ll have a framework like Qt. In which case, why not just use Qt from the start.

    In my opinion, the standard committee already pays too much attention to libraries, where I think it should mainly focus on improving the language itself and make it easier for library writers. They’ve been mostly copying Boost in to the standard anyway, i.e. fixing what need not be fixed. In the meanwhile, the community is waiting in vain for things that would bring tremendous value to everyone and things that could not be efficiently solved by libraries: static reflection, compile-time code execution and generation, co-routines, stable ABI, modules, proper algebraic types, etc.

  5. Could we maybe learn from the huge success of packages in the R community? The process of publishing packages in R is well-documented. It works well both for package developers (judged from the wealth of high quality packages) and users. R is cross-platform on the three major OSs. For C++ there might be an additional layer of complexity as make files would be needed (right?) compatible with major platforms.

  6. In my humble opinion the first addition should, or must, be some way to connect C++ to the internet. About GUI, I find very annoying to resort to additional libraries to that. Microsoft VC++ is limited (or inexistent for native C++) in that aspect (possibly Visual Code will make things easier in the future) and the best solution, for me, is QT (for GUI, for internet and so on…) but for static linking we need a comercial license.
    The solution to bring C++ to the XXI century, that you suggest, package manager, can be a good one. This would allow to maintain a contained standard library and solving all the problems and making everyone happy .

  7. This is way, way too late. The main reason I abandoned C++ and learned Java in the last century is that I could then write an entire application, I/O, screen output, desktop and web without having to buy and learn an expensive, proprietary set of libraries. My other reason was that the standard method of ending strings with a null, rather than preceding them with a character count was incredibly inefficient and, as we have found, open to bugs. After learning that the standard classes in Java required bending over backwards and writing ridiculous sets of inherited classes to do the simplest things, I abandoned Java and accepted the proprietary C# and the horribly named JavaScript, only going back to Java as necessary in the form of Android. So, if you ask me, I would take .NET as an example (not that even it can’t be improved) and go from there.

  8. I may not be alone in thinking that C++ has become the Assembly Language of this century. The use cases for a complete C++ solution remain but we should take care not to take C++ where it does not belong. Bytecode languages like Java and C# provide a next generation clarity that is clearly missing from C++. For all the things that you cannot do in Java and C# their dogmas do result in more maintainable and extensible code. I’m not saying that you cannot write bad code in Java or C# or that you cannot write good code in C++; but on the whole when I pick up 250,000 lines of code written by someone else — or myself more than 3 or 4 years ago, it’s easier to digest Java or C# than C++.

    I love C++ and I work in it almost every day but more and more I believe what I should be doing is restricting C++ to my lower level functionality. Functions that need to process billions of interdependent objects in real time for instance. ( I am currently working with prototype sensors collecting millions of events a second.) Once my code is talking to a database or interacting with a human being or some other activity that is glacial in comparison to my millions of events a second sensor it’s time to consider framing the problem in a Bytecode language.

    I’m currently working to wrap the better part of my C++ framework in CLR and C# libraries (and eventually SWIG to Java) so that it can be accessed from these more popular and arguably more easily maintainable languages. At university in the 70s I learned IBM 360 Assembler and subsequently x86 and DEC-PDP/VAX assemblers. It was a given that all one wrote is assembler were fragments: low level code that justified the use of a language without limits. Forty years later I’m feeling like I’ve come full circle.

    Yes, the Standard Library is too small. Much of what is in BOOST, but certainly not all, should become part of STD. But, C++ contains too much freedom and too few limitations to used in situations where it can be avoided. I need it for my time-is-of-the-essence applications and programming FPGA card and similar activities. But, I need to ask myself if each new task could not more reasonably be solved in a more modern OO language.

    Long live C++. It is the foundation for most of our programming languages; but let’s not live in the basement.

    1. I’m often surprised by how fast some C# applications can be, but I would also say that the counterargument to your argument is that CPU speeds aren’t getting anything but incrementally faster and a return to basics is necessary to understand how to get the most out of our logarithmically-decreasing speed curve.
      My opinion is that C++ needs to get over the whole backwards-compatibility argument (just use an older compiler!) and start ditching stuff. As to what, I not familiar enough with all areas to guess, but as an example, we don’t need different 5 ways to allocate memory – just ‘new’, with optional parameters such as ‘initialize/don’t-initialize’ and an allocator parameter.

  9. I agree with you. I believe C++ would be better if it had a built in graphics and GUI library kit. One of my attractions to C# was its integration with so many other built in libraries. It’s made the programming effort so much easier. There’s no reason why C++ shouldn’t have similar tools and libraries.

  10. I have a single board computer. For I/O it has a serial port and a single LED. If there is a C++ compiler for the processor on this board, I can probably get C++ programs to run. But wait, I can’t get the 2D graphics library to work, because I have no bitmapped display (OK, I have a single-bit monochrome bitmapped display).
    When people say a platofrm supports Java, they mean the whole thing, with its graphics libraries, networking, and all the rest. That means any platform that supports Java must have a display, network, etc. C++ is used in places that just don’t want to make that promise.
    If you want windowing, there’s vxWidgets, which is broadly supported. If you want low-level graphics, there’s OpenGL. These things don’t need to be part of the C++ standard.

  11. I’m looking forward to standard, cross-platform 2D graphics. Ok, there’s Boost and Cairo etc. but having ASIO & FileSystem in the standard library just makes life so much easier. It’s more work for the implementers but great for users.

  12. I agree with this, I also agree with the need for a 2D drawing library, I worry about the implementation however. There are so many pitfalls, it needs to be higher level than a 3D drawing library or it will fall into the Direct2D trap of being harder to use than 3D. It needs to mirror to a degree what is going on in the driver or it will become too slow to use effectively like almost all 2D drawing libraries. My current opinion is that by not taking an existing library or mirroring it closely . I very concerned that a shaderless implementation is beyond useless and a shaderful implementation indtroduces to much implementation detail to the standard.

    1. “My current opinion is that by not taking an existing library or mirroring it closely .”
      should read:
      My current opinion is that it should take an existing library or mirror it closely.

Leave a Reply

Your email address will not be published. Required fields are marked *