I now seperated the code into one shader I call bump, which only cares about the normals in the normal map for now. This should probably also look at the vertex normal too... but for now I only have it the terrain, so I guess it's ok.
The tree now only holds a phong shader, which use the model's normals to calculate the lighting.
I created a new terrain in L3DT that also has a normal map (but without baked in lights) to see the effects of the lighting model better. I need a detail texture with normalmap to get the effect out better, but right now the shading of the terrain is only a factor of the normalmap, as I don't use the terrain normal coords at all for the light calculation (fragment color).
The camera is now also slightly tilted, so that we can see more of the terrain. I will have a water plane cover some of the terrain, and then a skybox to surround everything nicely at the horizon.
