Skip to main content

How do you use templates?

Submitted by lorien on
Forum

Over the course of this year, my use of templates has completely changed: before this year I mostly just used the STL, and had written some specialised Ruby->C++ and C++->Ruby wrapping templates.

Late last year I started really studying templates. Turns out you have a whole new and different language inside of C++. The template system is actually a compile time scripting language. You can use it to calculate things while code is being compiled, and generate highly specialised code for particular variables whose value is known (or can be calculated) at compile time.

You can (in some compilers) write c++ code that throws the compiler into an infinite loop instantiating templates!

Something that really showed me how the template system actually works was in writing templatised replacements for the new and delete operators (create() and destroy()) to take advantage of the metaprogrammed memory allocator I posted here earlier this year.

I had to use the placement new operator, which you pass the memory you want to use for the class instance you are creating.
[code]
void* mem = malloc(sizeof(MyClass));
MyClass* myObj = new(mem) MyClass();
[/code]

placement new require you manually call destructors. You cannot use the normal delete operator.
[code]
myObj->~MyClass();
[/code]
My destroy function template looks something like this
[code]
template
void destroy(T* obj)
{
if(obj)
{
obj->~T();
free(obj);
}
}
[/code]
Though of course I use metaprogrammed allocation functions rather than malloc and free, so this exact code here is completely insane [:)]
The point of this code is one line:
[code] obj->~T() [/code]
The only way it can work is if the template scripting language works rather like the c preprocessor in that inside a template definition it goes around replacing "T" with the actual type you've placed in the <>'s.

I'd like to hear about interesting template stuff people are doing.

Submitted by Dragoon on Tue, 08/11/05 - 2:46 AM Permalink

Templates should be used only when they make sense (not often - they make sense for generic containers like lists, vectors, etc) or are a must (no choice), and avoided like the plague at all other times. The STL covers a good deal of what you need, and many features can be done through proper abstraction and design (cf design patterns).

They aren't very portable (yay for overly complex language features that are hard to implement), make debugging code harder, and increase compile time significantly. Maintenance (people unfamiliar with your code) is an order of magnitude worse with custom templates used throughout your code.

Generally they should be as short as possible, with as much functionality of what you are trying to achieve placed into normal classes as possible.

I'd be careful modifying your allocation / deallocation functions as it may make it harder to use 3rd party memory libraries and other debugging tools to trace memory issues and memory leaks.

The only time I've had a need to write some is for weak and strong smart pointers, counted pointers and auto pointers (and you can just use Boost for them). I also used some on mobile devices to implement math using arbitrary fixed point (eg. 12.4, 8.8, 4.12, 16.16, 12.20) base number classes without having to write vectors, matrices, math functions, etc for all variations on fixed we used.

Submitted by lorien on Tue, 08/11/05 - 4:52 AM Permalink

Thanks.

I saw some comments ago on flipcode about people thinking meta-programming is likely o make it's way in more and more engines. Not that that was what started me.

I came across the technique reading about ways scientific programmers use C++ for really heavy numeric computation. Then I found Blitz++ http://www.oonumerics.org/blitz/ and things started to make sense about why you might want to do some metaprogramming.

It's also part of boost (the boost MPL).

IMHO it is likely to be very usefull for extracting full performance in low level routines on the XBox360 and PS3 because branches are likely to be nasty and slow, and there is no branch prediction anymore. You can shift branches over to compile time.

I've been very careful- that's why I haven't overloaded new and delete globally, and I have my own leak tracking system built into the allocator now. Works rather like the one in MFC.

I have no arguemnt with a lot of what you said, I just think using it in low level details that get used over and over, and where the feature set isn't likely to change much isn't that big a deal.

Things like type lists can be pretty cool if you're trying to make a scripting language (I'm not anymore, I'm hacking one, but I spent a good 6 months of this year designing and coding one).

I'd say DON'T use anything like metaprogramming in game code. Use it (if you are going to) in low level engine code.

Submitted by redwyre on Sun, 20/11/05 - 12:33 PM Permalink

Templates aren't something to be afraid of, but - like just about every feature in C++ - they should be used when the situation calls for it. All the coders (interesting to note: especially the younger ones) I work with are fairly proficient with templates and metaprogramming concepts so I don't feel like I have to hold back in any metaprogramming. Though that said, I am not often presented with a situation in game code to use it.

The last thing I wrote was some templatised max functions that takes 8/4/2 values as template params and returns the biggest value at compile time. Though I do sometimes add templatised helper functions that handle casting internally which results in much simpler and cleaner code.

If you don't understand templates then I suggest you grab a book on the subject and learn, it's well worth it.

Submitted by Kezza on Mon, 21/11/05 - 4:35 AM Permalink

There are lots of little and handy uses for templates... don't just think that they're used only for overkill metaprogrammingmagic and container classes.
One really nice use for templates is templated abstract base classes/interfaces. Where applicable they can give you a base class that adheres to a fixed naming scheme but returns hard types, which saves alot of dynamic casting. It's not a technique without it's flaws but it's something that really grew on me.

Submitted by lorien on Wed, 23/11/05 - 6:24 AM Permalink

quote:Originally posted by Kezza
One really nice use for templates is templated abstract base classes/interfaces. Where applicable they can give you a base class that adheres to a fixed naming scheme but returns hard types, which saves alot of dynamic casting. It's not a technique without it's flaws but it's something that really grew on me.

While I get the general idea of what you're talking about could you perhaps post a little code? I don't think I've done that one before.

Edit: do you mean the Curiously Recursive Template pattern? It lets you do compile-time polymorphism of a sort.

Submitted by davidcoen on Wed, 30/11/05 - 5:18 AM Permalink

re: I'd like to hear about interesting template stuff people are doing.

the following is a rather old sample of some of my core vector/matrix library. was surpried by how to implement friend template functions outside the class definition (i like being able to read a class definition all on screen at once) and the compiler was getting upset at template paramterer reuse and resorting in function paramerter (example is the multiple function)

yes i'm being bad, working towards ease of code reuse rather than speed....

[code]
template class TMathMatrix;

template const TMathMatrix operator + (const TMathMatrix & in_lhs, const TYPE & in_rhs);

template const TMathMatrix operator * (const TMathMatrix & in_lhs, const TMathMatrix & in_rhs);

/**/
template class TMathMatrix
{
public:
TMathMatrix(const TMathMatrix & in_matrix);
TMathMatrix(const TYPE * in_data);
TMathMatrix(void);

friend const TMathMatrix operator + <> (const TMathMatrix & in_lhs, const TYPE & in_rhs);

template friend const TMathMatrix operator * (const TMathMatrix & in_lhs, const TMathMatrix & in_rhs);
};

/**/
template const TMathMatrix operator + (const TMathMatrix & in_lhs, const TYPE & in_rhs)
{
COMMON_CONDITION_PRE(COMMON_TRACE_PRIORITY_MATH_MATRIX)
{
}
TYPE data[WIDTH * HEIGHT];
s32 index;
s32 count;

count = WIDTH * HEIGHT;

for (index = 0; index < count; ++index)
{
data[index] = in_lhs.m_data[index] + in_rhs;
}

return TMathMatrix(data);
}

/**/
template const TMathMatrix operator * (const TMathMatrix & in_lhs, const TMathMatrix & in_rhs)
{
COMMON_CONDITION_PRE(COMMON_TRACE_PRIORITY_MATH_MATRIX)
{
}
TMathMatrix matrix;
const TYPE * lhs_data;
const TYPE * rhs_data;
s32 lhs_data_index;
s32 rhs_data_index;
s32 height_index;
s32 width_index;
s32 other_index;
s32 data_index;
TYPE value;

{
lhs_data = in_lhs.GetDataConst();
rhs_data = in_rhs.GetDataConst();
data_index = 0;
}

for (height_index = 0; height_index < HEIGHT; ++height_index)
{
for (width_index = 0; width_index < WIDTH; ++width_index)
{
value = 0;
lhs_data_index = width_index;
rhs_data_index = (height_index * IN_OTHER);
for (other_index = 0; other_index < IN_OTHER; ++other_index)
{
value += lhs_data[lhs_data_index] * rhs_data[rhs_data_index];
lhs_data_index += WIDTH;
rhs_data_index += 1;
}

matrix.m_data[data_index] = value;
++data_index;
}
}

return matrix;
}

[/code]

Submitted by lorien on Thu, 01/12/05 - 10:19 AM Permalink

hmm, not sure I see your point, to me that syntax seems plain though a little annoying- perhaps I've been reading too much perverted meta-code though [;)]

That's how templates are likely to be written more in future- as compilers begin to support the export keyword. The template bodies will be in a separate file, or compiled to some kind of byte-code. Afaik the language designers didn't realise how complicated the export keyword would be to impliment.

Nice use of templates to make a really flexible matrix btw. With a good compiler the for loops would likely get completely unrolled because you've used compile time constants.

I've started using a more class-like syntax for template declarations, this is my replacement for the new operator for types that take 4 constructor arguments for example.
[code]
template
<
class T,
typename C1,
typename C2,
typename C3,
typename C4
>
inline T* create(const char* file, int line, C1 c1, C2 c2, C3 c3, C4 c4)
{
void* mem = LDK::alloc(file,line);
try
{
return new (mem) T(c1,c2,c3,c4);
}
catch(std::exception& e)
{
LDK::dealloc(mem);
throw e;
}
catch(...)
{
LDK::dealloc(mem);
throw;
}
}

//////////////////////////////////////////////////////////////////////////////////////
/// def LDK_CREATE4(Type, arg1, arg2, arg3, arg4)
/// ingroup ObjectAlloc
/// rief create an object that takes 4 constructor arguments
/// hrow BadAlloc in an out of memory condition.
//////////////////////////////////////////////////////////////////////////////////////
#define LDK_CREATE4(Type,c1,c2,c3,c4) LDK::create(__FILE__,__LINE__,c1,c2,c3,c4)
[/code]

I use the preprocessor macros LDK_CREATEX because that's the only way I can think of to get the __FILE__ and __LINE__ macros to work for the site of the allocation. Anyone have any neat ideas on how to improve this? You can't overload preprocessor macros [:(]

Posted by lorien on
Forum

Over the course of this year, my use of templates has completely changed: before this year I mostly just used the STL, and had written some specialised Ruby->C++ and C++->Ruby wrapping templates.

Late last year I started really studying templates. Turns out you have a whole new and different language inside of C++. The template system is actually a compile time scripting language. You can use it to calculate things while code is being compiled, and generate highly specialised code for particular variables whose value is known (or can be calculated) at compile time.

You can (in some compilers) write c++ code that throws the compiler into an infinite loop instantiating templates!

Something that really showed me how the template system actually works was in writing templatised replacements for the new and delete operators (create() and destroy()) to take advantage of the metaprogrammed memory allocator I posted here earlier this year.

I had to use the placement new operator, which you pass the memory you want to use for the class instance you are creating.
[code]
void* mem = malloc(sizeof(MyClass));
MyClass* myObj = new(mem) MyClass();
[/code]

placement new require you manually call destructors. You cannot use the normal delete operator.
[code]
myObj->~MyClass();
[/code]
My destroy function template looks something like this
[code]
template
void destroy(T* obj)
{
if(obj)
{
obj->~T();
free(obj);
}
}
[/code]
Though of course I use metaprogrammed allocation functions rather than malloc and free, so this exact code here is completely insane [:)]
The point of this code is one line:
[code] obj->~T() [/code]
The only way it can work is if the template scripting language works rather like the c preprocessor in that inside a template definition it goes around replacing "T" with the actual type you've placed in the <>'s.

I'd like to hear about interesting template stuff people are doing.


Submitted by Dragoon on Tue, 08/11/05 - 2:46 AM Permalink

Templates should be used only when they make sense (not often - they make sense for generic containers like lists, vectors, etc) or are a must (no choice), and avoided like the plague at all other times. The STL covers a good deal of what you need, and many features can be done through proper abstraction and design (cf design patterns).

They aren't very portable (yay for overly complex language features that are hard to implement), make debugging code harder, and increase compile time significantly. Maintenance (people unfamiliar with your code) is an order of magnitude worse with custom templates used throughout your code.

Generally they should be as short as possible, with as much functionality of what you are trying to achieve placed into normal classes as possible.

I'd be careful modifying your allocation / deallocation functions as it may make it harder to use 3rd party memory libraries and other debugging tools to trace memory issues and memory leaks.

The only time I've had a need to write some is for weak and strong smart pointers, counted pointers and auto pointers (and you can just use Boost for them). I also used some on mobile devices to implement math using arbitrary fixed point (eg. 12.4, 8.8, 4.12, 16.16, 12.20) base number classes without having to write vectors, matrices, math functions, etc for all variations on fixed we used.

Submitted by lorien on Tue, 08/11/05 - 4:52 AM Permalink

Thanks.

I saw some comments ago on flipcode about people thinking meta-programming is likely o make it's way in more and more engines. Not that that was what started me.

I came across the technique reading about ways scientific programmers use C++ for really heavy numeric computation. Then I found Blitz++ http://www.oonumerics.org/blitz/ and things started to make sense about why you might want to do some metaprogramming.

It's also part of boost (the boost MPL).

IMHO it is likely to be very usefull for extracting full performance in low level routines on the XBox360 and PS3 because branches are likely to be nasty and slow, and there is no branch prediction anymore. You can shift branches over to compile time.

I've been very careful- that's why I haven't overloaded new and delete globally, and I have my own leak tracking system built into the allocator now. Works rather like the one in MFC.

I have no arguemnt with a lot of what you said, I just think using it in low level details that get used over and over, and where the feature set isn't likely to change much isn't that big a deal.

Things like type lists can be pretty cool if you're trying to make a scripting language (I'm not anymore, I'm hacking one, but I spent a good 6 months of this year designing and coding one).

I'd say DON'T use anything like metaprogramming in game code. Use it (if you are going to) in low level engine code.

Submitted by redwyre on Sun, 20/11/05 - 12:33 PM Permalink

Templates aren't something to be afraid of, but - like just about every feature in C++ - they should be used when the situation calls for it. All the coders (interesting to note: especially the younger ones) I work with are fairly proficient with templates and metaprogramming concepts so I don't feel like I have to hold back in any metaprogramming. Though that said, I am not often presented with a situation in game code to use it.

The last thing I wrote was some templatised max functions that takes 8/4/2 values as template params and returns the biggest value at compile time. Though I do sometimes add templatised helper functions that handle casting internally which results in much simpler and cleaner code.

If you don't understand templates then I suggest you grab a book on the subject and learn, it's well worth it.

Submitted by Kezza on Mon, 21/11/05 - 4:35 AM Permalink

There are lots of little and handy uses for templates... don't just think that they're used only for overkill metaprogrammingmagic and container classes.
One really nice use for templates is templated abstract base classes/interfaces. Where applicable they can give you a base class that adheres to a fixed naming scheme but returns hard types, which saves alot of dynamic casting. It's not a technique without it's flaws but it's something that really grew on me.

Submitted by lorien on Wed, 23/11/05 - 6:24 AM Permalink

quote:Originally posted by Kezza
One really nice use for templates is templated abstract base classes/interfaces. Where applicable they can give you a base class that adheres to a fixed naming scheme but returns hard types, which saves alot of dynamic casting. It's not a technique without it's flaws but it's something that really grew on me.

While I get the general idea of what you're talking about could you perhaps post a little code? I don't think I've done that one before.

Edit: do you mean the Curiously Recursive Template pattern? It lets you do compile-time polymorphism of a sort.

Submitted by davidcoen on Wed, 30/11/05 - 5:18 AM Permalink

re: I'd like to hear about interesting template stuff people are doing.

the following is a rather old sample of some of my core vector/matrix library. was surpried by how to implement friend template functions outside the class definition (i like being able to read a class definition all on screen at once) and the compiler was getting upset at template paramterer reuse and resorting in function paramerter (example is the multiple function)

yes i'm being bad, working towards ease of code reuse rather than speed....

[code]
template class TMathMatrix;

template const TMathMatrix operator + (const TMathMatrix & in_lhs, const TYPE & in_rhs);

template const TMathMatrix operator * (const TMathMatrix & in_lhs, const TMathMatrix & in_rhs);

/**/
template class TMathMatrix
{
public:
TMathMatrix(const TMathMatrix & in_matrix);
TMathMatrix(const TYPE * in_data);
TMathMatrix(void);

friend const TMathMatrix operator + <> (const TMathMatrix & in_lhs, const TYPE & in_rhs);

template friend const TMathMatrix operator * (const TMathMatrix & in_lhs, const TMathMatrix & in_rhs);
};

/**/
template const TMathMatrix operator + (const TMathMatrix & in_lhs, const TYPE & in_rhs)
{
COMMON_CONDITION_PRE(COMMON_TRACE_PRIORITY_MATH_MATRIX)
{
}
TYPE data[WIDTH * HEIGHT];
s32 index;
s32 count;

count = WIDTH * HEIGHT;

for (index = 0; index < count; ++index)
{
data[index] = in_lhs.m_data[index] + in_rhs;
}

return TMathMatrix(data);
}

/**/
template const TMathMatrix operator * (const TMathMatrix & in_lhs, const TMathMatrix & in_rhs)
{
COMMON_CONDITION_PRE(COMMON_TRACE_PRIORITY_MATH_MATRIX)
{
}
TMathMatrix matrix;
const TYPE * lhs_data;
const TYPE * rhs_data;
s32 lhs_data_index;
s32 rhs_data_index;
s32 height_index;
s32 width_index;
s32 other_index;
s32 data_index;
TYPE value;

{
lhs_data = in_lhs.GetDataConst();
rhs_data = in_rhs.GetDataConst();
data_index = 0;
}

for (height_index = 0; height_index < HEIGHT; ++height_index)
{
for (width_index = 0; width_index < WIDTH; ++width_index)
{
value = 0;
lhs_data_index = width_index;
rhs_data_index = (height_index * IN_OTHER);
for (other_index = 0; other_index < IN_OTHER; ++other_index)
{
value += lhs_data[lhs_data_index] * rhs_data[rhs_data_index];
lhs_data_index += WIDTH;
rhs_data_index += 1;
}

matrix.m_data[data_index] = value;
++data_index;
}
}

return matrix;
}

[/code]

Submitted by lorien on Thu, 01/12/05 - 10:19 AM Permalink

hmm, not sure I see your point, to me that syntax seems plain though a little annoying- perhaps I've been reading too much perverted meta-code though [;)]

That's how templates are likely to be written more in future- as compilers begin to support the export keyword. The template bodies will be in a separate file, or compiled to some kind of byte-code. Afaik the language designers didn't realise how complicated the export keyword would be to impliment.

Nice use of templates to make a really flexible matrix btw. With a good compiler the for loops would likely get completely unrolled because you've used compile time constants.

I've started using a more class-like syntax for template declarations, this is my replacement for the new operator for types that take 4 constructor arguments for example.
[code]
template
<
class T,
typename C1,
typename C2,
typename C3,
typename C4
>
inline T* create(const char* file, int line, C1 c1, C2 c2, C3 c3, C4 c4)
{
void* mem = LDK::alloc(file,line);
try
{
return new (mem) T(c1,c2,c3,c4);
}
catch(std::exception& e)
{
LDK::dealloc(mem);
throw e;
}
catch(...)
{
LDK::dealloc(mem);
throw;
}
}

//////////////////////////////////////////////////////////////////////////////////////
/// def LDK_CREATE4(Type, arg1, arg2, arg3, arg4)
/// ingroup ObjectAlloc
/// rief create an object that takes 4 constructor arguments
/// hrow BadAlloc in an out of memory condition.
//////////////////////////////////////////////////////////////////////////////////////
#define LDK_CREATE4(Type,c1,c2,c3,c4) LDK::create(__FILE__,__LINE__,c1,c2,c3,c4)
[/code]

I use the preprocessor macros LDK_CREATEX because that's the only way I can think of to get the __FILE__ and __LINE__ macros to work for the site of the allocation. Anyone have any neat ideas on how to improve this? You can't overload preprocessor macros [:(]