Addition of C++ std::string, std::vector to D

[gchatelet]

This is an early version serving as a basis for discussion since :

• there is no tests yet.
• it works only for std::string on Linux because of broken name mangling (no std::wstring, no std::vector).
• reverse iterator are not implemented.
• some functions are missing.

Also a few points to note :

mangling

• D ctor/dtor are not mangled as in C++, this need to be fixed.
• std::string defines operators to make working with string easier : =, ==, +=, operator bool. D cannot mangle them correctly for now.
• To prevent WTFs, STL containers are represented in D with value semantic (implemented as structs) but some platforms (eg. Windows) mangle struct and class differently. Since std::string is declared as a class in C++, D would not produce the expected mangling. One would need to add a #pragma(mangleAs, class).

string

• The current proposal replicates the C++ API for completeness. A simpler design could present std_string as a opaque type with a different, simpler API, basically getters/setters from D string.
• D struct ctor/dtor semantic conflicts with C++ semantic. Struct default ctor must be @disable so there's no way to construct an empty string apart from calling std::string(const char*) with empty string.

vector

• Basically just a skeleton for now (guarded by version(none)), it is not functional because of mangling issues.

dependency on libstdc++

• Addition of strings would require a dependency on libstdc++, we probably want to guard it behind a flag a la CRuntime_Glibc.
• Pragma(lib) looks inappropriate to specify libstdc++, Makefile ?

[ibuclaw]

Before I start off, I want to say this is genuinely an interesting PR.

[ibuclaw]

I believe you've already mentioned this, but all these pragma(mangle) declarations need to be removed. The compiler should mangle this for us, and if it doesn't, then raise a new issue. Someone will need to check compatibility with Windows too, which uses a completely different mangling ABI.

[gchatelet]

Yes they need to go away. The bug is already reported and I'm also working on getting this fixed - it's a lot harder than I thought though.

[redstar]

With the new extern(C++, class) and extern(C++, struct) syntax you should start an update of this PR. :-)

[gchatelet]

Thx for the ping @redstar. I'll update the PR within a few days.

[moon-chilled]

Any updates on this being merged?

[gchatelet]

@Elronnd I don't think this can be merged before the name mangling is fixed.
Unfortunately I spent a lot of time on trying to fix it and I came to the conclusion we have to reimplement what's done in clang. C++ name mangling is incredibly complicated as soon as you throw templates in.
I don't have time to work on this for now so it's basically stuck :-(
Any takers?

[timotheecour]

@gchatelet:
I was able to get good results using this with when adding #define _GLIBCXX_USE_CXX11_ABI 0 but wo this it doesn't wok; but what about supporting c++11 new string abi?
What would u recommend? (even a partial solution to get started)

[gchatelet]

I was not aware of a new ABI for c++ 11 strings. It might just be a name mangling issue. I suspect std::string is actually forwarded to another namespace.
From this stackoverflow it seems it's actually defined as std::__cxx11::string in GCC5 (WAT?).

We could make the namespace conditional but then the manual mangling needs to be changed as well. Anyways, the mangling should be fixed, the manual override is simply a hack for the proof of concept.

So if we want to be able to link against C++ libraries compiled with GCC5, we need something like:

version(GCC_CXX11_STRING) {
    extern(C++, std.__cxx11):
} else {
    extern(C++, std):
}

[JackStouffer]

@andralex Could you comment on the status of one of the scholarship students working on the STL interface?

[andralex]

cc @Darredevil

[timotheecour]

@gchatelet

It might just be a name mangling issue

unfortunately a bit more complex than that:

clang++ -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=0
sizeof(string):8

void temp_fun(string a){}
_Z8temp_funSs
temp_fun(std::basic_string<char, std::char_traits<char>, std::allocator<char> >)

string temp_fun2(){return "";}
_Z9temp_fun2v
temp_fun2()

clang++ -std=c++11:
sizeof(string):32 #NOTE: 32 instead of 8; cf short string optimization [1]

void temp_fun(string a){}
_Z8temp_funNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
temp_fun(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)
#NOTE: extra __cxx11::

string temp_fun2(){return "";}
_Z9temp_fun2B5cxx11v
temp_fun2[abi:cxx11]()  # NOTE extra [abi:cxx11] indicating the return type abi vs nothing before

[1]

• http://stackoverflow.com/questions/34560502/why-is-sizeofstdstring-only-eight-bytes
• http://blogs.msmvps.com/gdicanio/2016/11/17/the-small-string-optimization/

We could make the namespace conditional but then the manual mangling needs to be changed as well.

We definitely need:

• a version switch to specify ABI (cf D_GLIBCXX_USE_CXX11_ABI=0 or not)
• no code duplication on manual mangling; could achieve via a side function, eg:

string post_process_abi_mangling(string mangle){
version(D_GLIBCXX_USE_CXX11_ABI){
  // some logic here
}
else
  return mangle;
}

pragma(mangle, "_ZNKSs4sizeEv".post_process_abi_mangling)
foo;

[gchatelet]

@timotheecour

I believe the post_process_abi_mangling method won't work. Name mangling is much more complex than string manipulation: it depends on the compiler, the semantic of the scope symbols, the compression schemes (have a look at Itanium C++ ABI's mangling section for instance), I tried to understand how it works and summed it up, also this folder holds a bunch of tests for linux name mangling.

If there are several implementations (several sizes, behaviors) we need to make a call: support only the common case or have all the implementations and a way to select between them.

[PetarKirov]

You may be interested in this: http://fitzgeraldnick.com/2017/02/22/cpp-demangle.html

[gchatelet]

Thx for the article @ZombineDev. I'll add that demangling is easier than mangling because you don't need to deal with the impedance mismatch between C++ and D semantic (class is ref semantic in D, static arrays are passed as pointers in C++ just to name a few).

[mengu]

hi @gchatelet

are you still interested in getting this accepted? i'd appreciate it.

[gchatelet]

@mengu well unfortunately this is all blocked on the name mangling issue: and this is a tough one. I spent a lot of time on it already and now that I have a newborn... you know...
If someone wants to tackle it I can help, but my spare time is extremely small.

[wilzbach]

Superseded by #2211