- Compositor now is threaded
Enable it with the Scene buttons "Threads". This will handle over nodes to
individual threads to be calculated. However, if nodes depend on others
they have to wait. The current system only threads per entire node, not for
calculating results in parts.
I've reshuffled the node execution code to evaluate 'changed' events, and
prepare the entire tree to become simply parsed for open jobs with a call
to node = getExecutableNode()
By default, even without 'thread' option active, all node execution is
done within a separate thread.
Also fixed issues in yesterdays commit for 'event based' calculations, it
didn't do animated images, or execute (on rendering) the correct nodes
when you don't have Render-Result nodes included.
- Added generic Thread support in blenlib/ module
The renderer and the node system now both use same code for controlling the
threads. This has been moved to a new C file in blenlib/intern/threads.c.
Check this c file for an extensive doc and example how to use it.
The current implementation for Compositing allows unlimited amount of
threads. For rendering it is still tied to two threads, although it is
pretty easy to extend to 4 already. People with giant amounts of cpus can
poke me once for tests. :)
- Bugfix in creating group nodes
Group node definitions demand a clear separation of 'internal sockets' and
'external sockets'. The first are sockets being linked internally, the latter
are sockets exposed as sockets for the group itself.
When sockets were linked both internal and external, Blender crashed. It is
solved now by removing the external link(s).
Here are some notes written by Ton:
There's several areas I still have to work on, so don't immediately
report bugs!
- Preview renders in buttons window only supports materials now (no
lamp, texture or world)
- Yafray code has to be checked, it might not work
- unified render doesn't work, might not come back even
- lens flares dont work yet
- motion blur and field render doesnt work yet
- exr libraries are static linked still, we look at making it dynamic
loadable
- the compositor is fully in development still
I really advise people to check on the orange cvs logs too (in case you
ignored it), this has most of the information in it.
At this moment I still have to code for Orange deadlines, that defines
my priority listing mostly. And since we don't use Windows there, doing
testing and bugfixing for that platform would be of great help!
Best features of this commit to checkout:
- Grouping
- full fixed library linking system
- custom bone drawing
- bone layers
- Material/Shading nodes
- recode of render engine
- preview renders
- Compositing options
When the work pressure is a bit lower, I'll make nice docs for all of
this!
-Ton-
Here are my notes on things to look out for as potential problem
spots:
source/blender/blenkernel/intern/displist.c:
+ is initfastshade(void) supposed to be empty? I had
to make it empty to get the merged tree to compile.
source/blender/python/api2_2x/Armature.c:
+ went with the version that had Armature_getLayers()
source/blender/python/api2_2x/Object.c
+ went with the version of Object_getPose() from bf-blender.
(#ifdef 0-ed the other version)
source/blender/python/api2_2x/Pose.[ch]
+ had problems linking due to no Pose_Init() ... copied these
two files straight from bf-blender.
source/blender/src/drawview.c:
+ view3d_panel_properties() had things shifted a few things shifted
a few pixels, otherwise, things were painless
source/blender/src/splash.jpg.c:
+ went with bf-blender version (orange is dead)
source/gameengine:
+ went with bf-blender version -- does not compile due to IMB_rect* stuff,
Ton should look into this.
system tracking changes in nodes, making sure only these nodes and
the ones that depend, are executed.
Further the 'time cursor' now counts down to indicate which node is being
done.
Also: you now can disable the "use nodes" button in the header, edit all
changes, and when you press that button again it nicely executes the
changes.
Still on the todo:
- make compositing threaded
- find a way to nicely exit compositing on input events... so the UI
keeps being responsive
- idea; a 'percentage' menu in header to enforce calculations on smaller
images temporally
-> Rendering in RenderLayers
It's important to distinguish a 'render layer' from a 'pass'. The first is
control over the main pipeline itself, to indicate what geometry is being
is rendered. The 'pass' (not in this commit!) is related to internal
shading code, like shadow/spec/AO/normals/etc.
Options for RenderLayers now are:
- Indicate which 3d 'view layers' have to be included (so you can render
front and back separately)
- "Solid", all solid faces, includes sky at the moment too
- "ZTransp", all transparent faces
- "Halo", the halos
- "Strand", the particle strands (not coded yet...)
Currently only 2 'passes' are exported for render, which is the "Combined"
buffer and the "Z. The latter now works, and can be turned on/off.
Note that all layers are still fully kept in memory now, saving the tiles
and layers to disk (in exr) is also todo.
-> New Blur options
The existing Blur Node (compositor) now has an optional input image. This
has to be a 'value buffer', which can be a Zbuffer, or any mask you can
think of. The input values have to be in the 0-1 range, so another new
node was added too "Map Value".
The value input can also be used to tweak blur size with the (todo)
Time Node.
Temporal screenies:
http://www.blender.org/bf/rt.jpghttp://www.blender.org/bf/rt1.jpghttp://www.blender.org/bf/rt2.jpg
BTW: The compositor is very slow still, it recalulates all nodes on each
change still. Persistant memory and dependency checks is coming!
Image Node now supports animations as well. Press the 'Sequence' Icon in
the node to see the options:
- Frs: the amount of images involved in the sequence
- SFra: start frame, the Blender frame number
- First: the number in the name of the first image in the sequence
- Cycl: make the sequence go cyclic
The images are refreshed on each call to execute the node tree, but not
on advancing frames in Blender... I suspect that might make it to slow now,
will first have to code nice threaded/optimized updates in compositor.
- New Node "Composite" is output node that puts composited result back
in render pipeline.
- This then also displays in the render window while editing
- But, only with Scene buttons option "Do Compositor" set
- Then, just press F12 or render anims to see the magic!
For clarity, the former 'Output" node is renamed to "Viewer".
- Curves in Image window didn't copy alpha (when no 32 bits rect for
display was made already).
- Code was missing to validate grouped nodes in Compositor, didnt work
at all yet
- Loading new Images in a Node (using filewindow) didn't assign the image
Image window now displays Alpha for float images too
- Image curves are only applied now when curves panel is in use. Closing
the panel will disable curves, reopen Panel enable curves.
You can minimize the Curves panel to keep it active. That latter then
works as visualization curves are being applied.
- Compositor output now also uses Image window curves, if active.
A full detailed description of this will be done later... is several days
of work. Here's a summary:
Render:
- Full cleanup of render code, removing *all* globals and bad level calls
all over blender. Render module is now not called abusive anymore
- API-fied calls to rendering
- Full recode of internal render pipeline. Is now rendering tiles by
default, prepared for much smarter 'bucket' render later.
- Each thread now can render a full part
- Renders were tested with 4 threads, goes fine, apart from some lookup
tables in softshadow and AO still
- Rendering is prepared to do multiple layers and passes
- No single 32 bits trick in render code anymore, all 100% floats now.
Writing images/movies
- moved writing images to blender kernel (bye bye 'schrijfplaatje'!)
- made a new Movie handle system, also in kernel. This will enable much
easier use of movies in Blender
PreviewRender:
- Using new render API, previewrender (in buttons) now uses regular render
code to generate images.
- new datafile 'preview.blend.c' has the preview scenes in it
- previews get rendered in exact displayed size (1 pixel = 1 pixel)
3D Preview render
- new; press Pkey in 3d window, for a panel that continuously renders
(pkey is for games, i know... but we dont do that in orange now!)
- this render works nearly identical to buttons-preview render, so it stops
rendering on any event (mouse, keyboard, etc)
- on moving/scaling the panel, the render code doesn't recreate all geometry
- same for shifting/panning view
- all other operations (now) regenerate the full render database still.
- this is WIP... but big fun, especially for simple scenes!
Compositor
- Using same node system as now in use for shaders, you can composit images
- works pretty straightforward... needs much more options/tools and integration
with rendering still
- is not threaded yet, nor is so smart to only recalculate changes... will be
done soon!
- the "Render Result" node will get all layers/passes as output sockets
- The "Output" node renders to a builtin image, which you can view in the Image
window. (yes, output nodes to render-result, and to files, is on the list!)
The Bad News
- "Unified Render" is removed. It might come back in some stage, but this
system should be built from scratch. I can't really understand this code...
I expect it is not much needed, especially with advanced layer/passes
control
- Panorama render, Field render, Motion blur, is not coded yet... (I had to
recode every single feature in render, so...!)
- Lens Flare is also not back... needs total revision, might become composit
effect though (using zbuffer for visibility)
- Part render is gone! (well, thats obvious, its default now).
- The render window is only restored with limited functionality... I am going
to check first the option to render to a Image window, so Blender can become
a true single-window application. :)
For example, the 'Spare render buffer' (jkey) doesnt work.
- Render with border, now default creates a smaller image
- No zbuffers are written yet... on the todo!
- Scons files and MSVC will need work to get compiling again
OK... thats what I can quickly recall. Now go compiling!
This is using instructions from Ton, so hopefully the implementation is ok.
This is really needed here where we are using all sorts of wacky scales, and
empties look too big or too small. Of course we don't want to scale the
empties because there are often things parented to them.
New options are in edit buttons for empties to control the display style
and the size. New styles are easy to add, too. Just needs useful ideas and
minor effort from anyone who wants to.
Support for copying these values has also been added to the Copy Attributes
->Drawtype menu command.
- cleanup of color curves code; goes at least twice faster now!
(includes black/white point stuff)
- When using 'Curves' in image window on a byte rect, it creates a (temp)
float rect to operate on. So curves work for regular pictures too now.
- adds object.getPose
- ability to manipulate poses /posebones
- fixes a overflow bug in matrix sequence accessor
- adds code to get vec/roll from mat3
- few internal fixes to NLA
- ability to set bone matrices
- EXR now saves and reads Zbuffers correctly
- EXR reading didn't set alpha to 1 yet when no alpha buffer was present
- ImageWindow: the "black point" only checked for the r value... now is OK
- ImageWindow: Curves panal has button "reset"
- ImageWindow: hold LMB drag shows rgba and z values. With SHIFT or CTRL it
applies black/white point whilte dragging too
- ImageWindow: saving file copied the entire buffer... removed that. Also
made the header print clear; this save only saves in own file type.
- Curves: zoom and drag now gets clamped by the Clipping value
- Imbuf: duplicate buffer only copied one quarter of to new buffer
This fixes most of the UV distortion issues with subsurf. Near seams
however there might still be some distortion, but this should at least
not be worse than before. Subsurf UV is enabled by default on new meshes,
and can be enabled in the modifier panel for existing ones.
Before and after:
http://users.pandora.be/blendix/notsmooth.pnghttp://users.pandora.be/blendix/smooth.png
Image as loaded in Blender (from openexr.com):
http://www.blender.org/bf/exrcurve1.jpg
Image with different white point:
http://www.blender.org/bf/exrcurve2.jpg
Image with white and black point and a curve:
http://www.blender.org/bf/exrcurve3.jpg
Use SHIFT+click to set the black point, and CTRL+click for white point.
The buttons in the panel work too, of course.
The curves work after the black/white range was corrected, so you can
stick to curves with a normal 0-1 range.
There's also now a general color curve, marked with 'C' button.
Note; this currently only maps the float colors to a visible 8 bits per
channel rect. You can save it, but when the blender file loads the curve
or mapping is not executed until you click in the curves... have to look
at that still.
Speed for this is also quite unoptimized... still WIP, but fun!
- Reading exr images now goes OK. I've unified the code for reading
'half' and 'float' (was nicely possible!). And removed useless copying
of data around.
- Fixed bug in allocating new rects, like for making mipmaps. flag issues.
- filter code accidentally incremented wrong pointer (crash on mipmap too)
I also provided a fix for tiff images. (It was appending tif even if
it was already in the name, now it does it correctly)
(no more image.tif.tif)
Kent
Credits go to Gernot Ziegler, who originally coded EXR support, and to
Austin Benesh for bringing it further. Kent Mein provided a lot of code
for integrating float buffers in Blender imbuf and ImBuf API cleanup,
and provided Make and Scons and static linking.
At this moment; the EXR libraries are a *dependency*, so you cannot get
the Orange branch compiled without having OpenEXR installed. Get the
(precompiled or sources) stuff from www.openexr.com. Current default is
that the headers and lib resides in /user/local/
Several changes/additions/fixes were added:
- EXR code only supported 'half' format (16 bits per channel). I've added
float writing, but for reading it I need tomorrow. :)
- Quite some clumsy copying of data happened in EXR code.
- cleaned up the api calls already a bit, preparing for more advanced
support
- Zbuffers were saved 16 bits, now 32 bits
- automatic adding of .exr extensions went wrong
Imbuf:
- added proper imbuf->flags and imbuf->mall support for float buffers, it
was created for *each* imbuf. :)
- found bugs for float buffers in scaling and flipping. Code there will
need more checks still
- imbuf also needs to be verified to behave properly when no 32 bits
rect exists (for saving for example)
TODO:
- support internal float images for textures, backbuf, AO probes, and
display in Image window
Hope this commit won't screwup syncing with bf-blender... :/
- New UI element: the "Curve Button".
For mapping ranges (like 0 - 1) to another range, the curve button can be
used for proportional falloff, bone influences, painting density, etc.
Most evident use is of course to map RGB color with curves.
To be able to use it, you have to allocate a CurveMapping struct and pass
this on to the button. The CurveMapping API is in the new C file
blenkernel/intern/colortools.c
It's as simple as calling:
curvemap= curvemapping_add(3, 0, 0, 1, 1)
Which will create 3 curves, and sets a default 0-1 range. The current code
only supports up to 4 curves maximum per mapping struct.
The CurveMap button in Blender than handles allmost all editing.
Evaluating a single channel:
float newvalue= curvemapping_evaluateF(curvemap, 0, oldval);
Where the second argument is the channel index, here 0-1-2 are possible.
Or mapping a vector:
curvemapping_evaluate3F(curvemap, newvec, oldvec);
Optimized versions for byte or short mapping is possible too, not done yet.
In butspace.c I've added a template wrapper for buttons around the curve, to
reveil settings or show tools; check this screenie:
http://www.blender.org/bf/curves.jpg
- Buttons R, G, B: select channel
- icons + and -: zoom in, out
- icon 'wrench': menu with tools, like clear curve, set handle type
- icon 'clipping': menu with clip values, and to dis/enable clipping
- icon 'x': delete selection
In the curve button itself, only LMB clicks are handled (like all UI elements
in Blender).
- click on point: select
- shift+click on point: swap select
- click on point + drag: select point (if not selected) and move it
- click outside point + drag: translate view
- CTRL+click: add new point
- hold SHIFT while dragging to snap to grid
(Yes I know... either one of these can be Blender compliant, not both!)
- if you drag a point exactly on top of another, it merges them
Other fixes:
- Icons now draw using "Safe RasterPos", so they align with pixel boundary.
the old code made ints from the raster pos coordinate, which doesn't work
well for zoom in/out situations
- bug in Node editing: buttons could not get freed, causing in memory error
prints at end of a Blender session. That one was a very simple, but nasty
error causing me all evening last night to find!
(Hint; check diff of editnode.c, where uiDoButtons is called)
Last note: this adds 3 new files in our tree, I did scons, but not MSVC!
http://projects.blender.org/tracker/?func=detail&atid=125&aid=3658&group_id=9
Running scripts from command line in bg mode:
blender -b -P myscript.py
crashes Blender 2.40.
The problem is in add_text() in text.c: G.scene can be NULL at this
point (in bg mode). Added a check:
line 323:
if (G.scene) /* can be NULL (bg mode) */
BLI_convertstringcode(str, G.sce, G.scene->r.cfra);
The text being added with add_text() in this particular case is the
script filename specified at the command prompt, so it should be ok to skip
BLI_convertstringcode. Feel free to disagree, though.
- Adds missing ipos for texture
* color, contrast, brightness
- Adds IKey menu in the texture subcontext
#3711 (bug tracker)
- report and patch for missing blend types Halo and Radial
Patches by Levon Hudson
Armatures are back
Split screen
Double sided lightning
Ambient lighting
Alpha test
Material IPO support (one per object atm)
Blender materials
GLSL shaders - Python access
Up to three texture samplers from the material panel ( 2D & Cube map )
Python access to a second set of uv coordinates
See http://www.elysiun.com/forum/viewtopic.php?t=58057
- removed debugging output from fluidsim export
- directores with "+" are now valid for fluidsim data
- simulation now always uses frame 1 to endframe, so changing start frame settings should work again
- New Node: "Mapping". Allows input vector to be translated, rotated and
scaled. And optional be clipped to a range. Works for colors too!
- The button "Normal" now allows incremental input, so a click in the
button won't change the normal anymore
- Connecting wires now show selection state for Nodes, with nice blended
colors. Both colors were added in Themes, but default to black and white
guitargeek), this commit enhances the support for temporary storage
for the structs EditVert, EditEdge, and EditFace. The field
"EditVert *vn" has been removed and replaced by a union called
"tmp" that can hold:
v, an EditVert pointer;
e, an EditEdge pointer;
f, an EditFace pointer;
fp, a float pointer;
p, a void pointer;
l, a long;
Please see the mailing list post here for more information about
this:
http://projects.blender.org/pipermail/bf-committers/2005-December/012877.html
- Previews inside groups now get updated too
- Activating nodes inside of groups updates UI and preview render correctly
- Entering/leaving groups updates UI and previewrender
- Material Node: now draws socket name next to colorpicker for inputs
Material Nodes now have input sockets, to override the actual Material
settings. If socket has no input, you can also edit Material settings here.
(Color picker, number slider).
Of course a load of options will become available here, first bugfixing!
(Oh, and enable fix thread render)
the coordinate outputs now have correct dx/dy vectors for Image AA, and
texture delivers correct intensity, rgb, alpha and normal.
Note; we need a "Vector Mapping" node, to do 2d/3d mapping, like in the
Material "Map In" panel.
**** NEW: Group Nodes
Node trees usually become messy and confusing quickly, so we need
not only a way to collapse Nodes into single 'groups', but also a
way to re-use that data to create libraries of effects.
This has been done by making a new Library data type, the NodeTree.
Everything that has been grouped is stored here, and available for
re-use, appending or linking. These NodeTrees are fully generic,
i.e. can store shader trees, composit trees, and so on. The 'type'
value as stored in the NodeTree will keep track of internal type
definitions and execute/drawing callbacks. Needless to say, re-using
shader trees in a composit tree is a bit useless, and will be
prevented in the browsing code. :)
So; any NodeTree can become a "Goup Node" inside in a NodeTree. This
Group Node then works just like any Node.
To prevent the current code to become too complex, I've disabled
the possibility to insert Groups inside of Groups. That might be
enabled later, but is a real nasty piece of code to get OK.
Since Group Nodes are a dynamic Node type, a lot of work has been
done to ensure Node definitions can be dynamic too, but still allow
to be stored in files, and allow to be verified for type-definition
changes on reloading. This system needs a little bit maturing still,
so the Python gurus should better wait a little bit! (Also for me to
write the definite API docs for it).
What works now:
- Press CTRL+G to create a new Group. The grouping code checks for
impossible selections (like an unselected node between selected nodes).
Everthing that's selected then gets removed from the current tree, and
inserted in a new NodeTree library data block. A Group Node then is
added which links to this new NodeTree.
- Press ALT+G to ungroup. This will not delete the NodeTree library
data, but just duplicate the Group into the current tree.
- Press TAB, or click on the NodeTree icon to edit Groups. Note that
NodeTrees are instances, so editing one Group will also change the
other users.
This also means that when removing nodes in a Group (or hiding sockets
or changing internal links) this is immediately corrected for all users
of this Group, also in other Materials.
- While editing Groups, only the internal Nodes can be edited. A single
click outside of the Group boundary will close this 'edit mode'.
What needs to be done:
- SHIFT+A menu in toolbox style, also including a list of Groups
- Enable the single-user button in the Group Node
- Displaying all (visible) internal group UI elements in the Node Panel
- Enable Library linking and prevent editing of Groups then.
**** NEW: Socket Visibility control
Node types will be generated with a lot of possible inputs or outputs,
and drawing all sockets all the time isn't very useful then.
A new option in the Node header ('plus' icon) allows to either hide all
unused sockets (first keypress) or to reveil them (when there are hidden
sockets, the icon displays black, otherwise it's blended).
Hidden sockets in Nodes also are not exported to a Group, so this way
you can control what options (in/outputs) exactly are available.
To be done:
- a way to hide individual sockets, like with a RMB click on it.
**** NEW: Nodes now render!
This is still quite primitive, more on a level to replace the (now
obsolete and disabled) Material Layers.
What needs to be done:
- make the "Geometry" node work properly, also for AA textures
- make the Texture Node work (does very little at the moment)
- give Material Nodes all inputs as needed (like Map-to Panel)
- find a way to export more data from a Material Node, like the
shadow value, or light intensity only, etc
Very important also to separate from the Material Buttons the
"global" options, like "Ztransp" or "Wire" or "Halo". These can not
be set for each Material-Node individually.
Also note that the Preview Render (Buttons window) now renders a bit
differently. This was a horrid piece of antique code, using a totally
incompatible way of rendering. Target is to fully re-use internal
render code for previews.
OK... that's it mostly. Now test!
