Added WM Jobs manager
- WM can manage threaded jobs for you; just provide a couple
of components to get it work:
- customdata, free callback for it
- timer step, notifier code
- start callback, update callback
- Once started, each job runs an own timer, and will for
every time step check necessary updates, or close the
job when ready.
- No drawing happens in jobs, that's for notifiers!
- Every job stores an owner pointer, and based on this owner
it will prevent multiple jobs to enter the stack.
Instead it will re-use a running job, signal it to stop
and allow caller to re-initialize it even.
- Check new wm_jobs.c for more explanation. Jobs API is still
under construction.
Fun: BLI_addtail(&wm->jobs, steve); :)
Put Node shader previews back using wmJobs
- Preview calculating is now fully threaded (1 thread still)
- Thanks to new event system + notifiers, you can see
previews update even while dragging sliders!
- Currently it only starts when you change a node setting.
Warning: the thread render shares Node data, so don't delete
nodes while it renders! This topic is on the todo to make safe.
Also:
- bug in region initialize (do_versions) showed channel list in
node editor wrong.
- flagged the channel list 'hidden' now, it was really in the
way! This is for later to work on anyway.
- recoded Render API callbacks so it gets handlers passed on,
no globals to use anymore, remember?
- previewrender code gets now so much nicer! Will remove a lot
of stuff from code soon.
Think global, act local!
The old favorite G.scene gone! Man... that took almost 2 days.
Also removed G.curscreen and G.edbo.
Not everything could get solved; here's some notes.
- modifiers now store current scene in ModifierData. This is not
meant for permanent, but it can probably stick there until we
cleaned the anim system and depsgraph to cope better with
timing issues.
- Game engine G.scene should become an argument for staring it.
Didn't solve this yet.
- Texture nodes should get scene cfra, but the current implementation
is too tightly wrapped to do it easily.
From the anti-globalization department:
G.obedit terminated!
Wherever possible, use CTX_data_edit_object(C) to get this
now. It's stored in scene now, and the screen context has
it defined.
* DNA_radio_types.h: done. (patch by Jorge Bernal). Also adds
some #defines in the radiosity DNA since it was using hardcoded
values.
* Added an "UnknownType" which has no properties, useful as a
placeholder for pointers that have no defined type yet.
* Sort a few lists in the code alphabetically.
svn merge https://svn.blender.org/svnroot/bf-blender/trunk/blender -r12987:17416
Issues:
* GHOST/X11 had conflicting changes. Some code was added in 2.5, which was
later added in trunk also, but reverted partially, specifically revision
16683. I have left out this reversion in the 2.5 branch since I think it is
needed there.
http://projects.blender.org/plugins/scmsvn/viewcvs.php?view=rev&root=bf-blender&revision=16683
* Scons had various conflicting changes, I decided to go with trunk version
for everything except priorities and some library renaming.
* In creator.c, there were various fixes and fixes for fixes related to the -w
-W and -p options. In 2.5 -w and -W is not coded yet, and -p is done
differently. Since this is changed so much, and I don't think those fixes
would be needed in 2.5, I've left them out.
* Also in creator.c: there was code for a python bugfix where the screen was not
initialized when running with -P. The code that initializes the screen there
I had to disable, that can't work in 2.5 anymore but left it commented as a
reminder.
Further I had to disable some new function calls. using src/ and python/, as
was done already in this branch, disabled function calls:
* bpath.c: error reporting
* BME_conversions.c: editmesh conversion functions.
* SHD_dynamic: disabled almost completely, there is no python/.
* KX_PythonInit.cpp and Ketsji/ build files: Mathutils is not there, disabled.
* text.c: clipboard copy call.
* object.c: OB_SUPPORT_MATERIAL.
* DerivedMesh.c and subsurf_ccg, stipple_quarttone.
Still to be done:
* Go over files and functions that were moved to a different location but could
still use changes that were done in trunk.
* subsurf code had a lot of unused variables, removed these where they are obviously not needed. commented if they could be useful later.
* some variables declorations hide existing variables (many of these left), but fixed some that could cause confusion.
* removed unused vars
* obscure python memory leak with colorband.
* make_sample_tables had a loop running wasnt used.
* if 0'd functions in arithb.c that are not used yet.
* made many functions static
the features that are needed to run the game. Compile tested with
scons, make, but not cmake, that seems to have an issue not related
to these changes. The changes include:
* GLSL support in the viewport and game engine, enable in the game
menu in textured draw mode.
* Synced and merged part of the duplicated blender and gameengine/
gameplayer drawing code.
* Further refactoring of game engine drawing code, especially mesh
storage changed a lot.
* Optimizations in game engine armatures to avoid recomputations.
* A python function to get the framerate estimate in game.
* An option take object color into account in materials.
* An option to restrict shadow casters to a lamp's layers.
* Increase from 10 to 18 texture slots for materials, lamps, word.
An extra texture slot shows up once the last slot is used.
* Memory limit for undo, not enabled by default yet because it
needs the .B.blend to be changed.
* Multiple undo for image painting.
* An offset for dupligroups, so not all objects in a group have to
be at the origin.
from Roelf De Kock (kiemdoder)
Fixes bug [#7969] Mirroring Object Breaks Radiosity Calculations
- copied from the tracker.
The code in this patch detects whether an object has negative scale (test the OB_NEG_SCALE bit in Object.transflag)
and then compensate for the negative scale when the surface normals are calculated for a radiosity render.
* tweak other SConscripts for editors/include
* commented BKE_bad_level_calls.h in writeframeserver.c - please check it doesn't break make builds
This compiles until final linking, which still fails.
=================
Big commit, but little user visible changes.
- Dupliverts and duplifaces are now rendered as instances, instead
of storing all of the geometry for each dupli, now an instance is
created with a matrix transform refering to the source object.
This should allow us to render tree leaves more memory efficient.
- Radiosity and to some degree raytracing of such objects is not
really efficient still. For radiosity this is fundamentally hard
to solve, but raytracing an octree could be created for each object,
but the current octree code with it's fixed size doesn't allow this
efficiently.
- The regression tests survived, but with I expect that some bugs will
pop up .. hopefully not too many :).
Implementation Notes
====================
- Dupligroups and linked meshes are not rendered as instances yet,
since they can in fact be different due to various reasons,
instancing of these types of duplis that are the same can be added
for them at a later point.
- Each ObjectRen now stores it's own database, instead of there being
one big databases of faces, verts, .. . Which objects that are actually
rendered are defined by the list of ObjectRenInstances, which all refer
to an ObjectRen.
- Homogeneous coordinatess and clipping is now not stored in vertices
anymore, but instead computed on the fly. This couldn't work for
instances. That does mean some extra computation has to be done, but
memory lookups can be slow too, and this saves some memory. Overall
I didn't find a significant speed impact.
- OSA rendering for solid and ztransp now is different. Instead of e.g.
going 8 times over the databases times and rendering the z-buffer, it
now goes over the database once and renders each polygon 8 times. That
was necessary to keep instances efficient, and can also give some
performance improvement without instances.
- There was already instancing support in the yafray export code, now it
uses Blender's render instances for export.
- UV and color layer storage in the render was a bit messy before, now
should be easier to understand.
- convertblender.c was reorganized somewhat. Regular render, speedvector
and baking now use a single function to create the database, previously
there was code duplicated for it.
- Some of these changes were done with future multithreading of scene
and shadow buffer creation in mind, though especially for scene creation
much work remains to be done to make it threadsafe, since it also involves
a lot of code from blenkernel, and there is an ugly conflict with the way
dupli groups work here .. though in the render code itself it's almost there.
Added support for multiple UVs in the render engine. This also involved
changing the way faces are stored, to allow data to be added optionally
per 256 faces, same as the existing system for vertices.
A UV layer can be specified in the Map Input panel and the Geometry node
by name. Leaving this field blank will default to the active UV layer.
Also added sharing of face selection and hiding between UV layers, and at
the same time improved syncing with editmode selection and hiding.
Still to do:
- Multi UV support for fastshade.
- Multires and NMesh preservation of multiple UV sets.
These can be created and deleted in the Mesh panel in the same place as
before. There is always one active UV and vertex color layer, that is
edited and displayed.
Important things to do:
- Render engine, material support
- Multires and NMesh now lose non active layers
Also CustomData changes to support muliple layers of the same type, and
changes to layer allocation, updated documentation is here:
http://mediawiki.blender.org/index.php/BlenderDev/BlenderArchitecture/CustomData
All data layers, including MVert/MEdge/MFace, are now managed as custom
data layers. The pointers like Mesh.mvert, Mesh.dvert or Mesh.mcol are
still used of course, but allocating, copying or freeing these arrays
should be done through the CustomData API.
Work in progress documentation on this is here:
http://mediawiki.blender.org/index.php/BlenderDev/BlenderArchitecture/CustomData
Replaced TFace by MTFace:
This is the same struct, except that it does not contain color, that now
always stays separated in MCol. This was not a good design decision to
begin with, and it is needed for adding multiple color layers later. Note
that this does mean older Blender versions will not be able to read UV
coordinates from the next release, due to an SDNA limitation.
Removed DispListMesh:
This now fully replaced by DerivedMesh. To provide access to arrays of
vertices, edges and faces, like DispListMesh does. The semantics of the
DerivedMesh.getVertArray() and similar functions were changed to return
a pointer to an array if one exists, or otherwise allocate a temporary
one. On releasing the DerivedMesh, this temporary array will be removed
automatically.
Removed ssDM and meshDM DerivedMesh backends:
The ssDM backend was for DispListMesh, so that became obsolete automatically.
The meshDM backend was replaced by the custom data backend, that now figures
out which layers need to be modified, and only duplicates those.
This changes code in many places, and overall removes 2514 lines of code.
So, there's a good chance this might break some stuff, although I've been
testing it for a few days now. The good news is, adding multiple color and
uv layers should now become easy.
* This commit is all of the rewrite work done on the SCons system. For
documentation see doc/blender-scons.txt and doc/blender-scons-dev.txt.
Also http://mediawiki.blender.org/index.php/BlenderDev/SconsRefactoring
contains valuable information, along with what still needs to be done.
- linux, os x and windows compile now.
- files are compiled to BF_INSTALLDIR (see config/(platform)-config.py)
- NOTE: Jean-Luc P will commit sometime during the weekend proper
appit() for OS X. For now, copy the resulting binary to an
existing .app bundle.
- features:
- cleaner structure for better maintenance
- cleaner output during compile
- better handling of build options
- general overall speed increase
- see the wiki for more info
Cygwin, FreeBSD and Solaris systems still need work. For these systems:
1) copy a config/(platform)-config.py to ie. config/cygwin-config.py
2) set the proper defaults for your platform
3) mail me at jesterking at letwory dot net with you configuration. if
you need any modifications to the system, do send a patch, too.
I'll be giving first-aid today and tomorrow, after that it'll be all
regular development work :)
/Nathan
A full detailed description of this will be done later... is several days
of work. Here's a summary:
Render:
- Full cleanup of render code, removing *all* globals and bad level calls
all over blender. Render module is now not called abusive anymore
- API-fied calls to rendering
- Full recode of internal render pipeline. Is now rendering tiles by
default, prepared for much smarter 'bucket' render later.
- Each thread now can render a full part
- Renders were tested with 4 threads, goes fine, apart from some lookup
tables in softshadow and AO still
- Rendering is prepared to do multiple layers and passes
- No single 32 bits trick in render code anymore, all 100% floats now.
Writing images/movies
- moved writing images to blender kernel (bye bye 'schrijfplaatje'!)
- made a new Movie handle system, also in kernel. This will enable much
easier use of movies in Blender
PreviewRender:
- Using new render API, previewrender (in buttons) now uses regular render
code to generate images.
- new datafile 'preview.blend.c' has the preview scenes in it
- previews get rendered in exact displayed size (1 pixel = 1 pixel)
3D Preview render
- new; press Pkey in 3d window, for a panel that continuously renders
(pkey is for games, i know... but we dont do that in orange now!)
- this render works nearly identical to buttons-preview render, so it stops
rendering on any event (mouse, keyboard, etc)
- on moving/scaling the panel, the render code doesn't recreate all geometry
- same for shifting/panning view
- all other operations (now) regenerate the full render database still.
- this is WIP... but big fun, especially for simple scenes!
Compositor
- Using same node system as now in use for shaders, you can composit images
- works pretty straightforward... needs much more options/tools and integration
with rendering still
- is not threaded yet, nor is so smart to only recalculate changes... will be
done soon!
- the "Render Result" node will get all layers/passes as output sockets
- The "Output" node renders to a builtin image, which you can view in the Image
window. (yes, output nodes to render-result, and to files, is on the list!)
The Bad News
- "Unified Render" is removed. It might come back in some stage, but this
system should be built from scratch. I can't really understand this code...
I expect it is not much needed, especially with advanced layer/passes
control
- Panorama render, Field render, Motion blur, is not coded yet... (I had to
recode every single feature in render, so...!)
- Lens Flare is also not back... needs total revision, might become composit
effect though (using zbuffer for visibility)
- Part render is gone! (well, thats obvious, its default now).
- The render window is only restored with limited functionality... I am going
to check first the option to render to a Image window, so Blender can become
a true single-window application. :)
For example, the 'Spare render buffer' (jkey) doesnt work.
- Render with border, now default creates a smaller image
- No zbuffers are written yet... on the todo!
- Scons files and MSVC will need work to get compiling again
OK... thats what I can quickly recall. Now go compiling!
(As usual movies disappears after while)
Face example showing stress values on a blend. White is stretch, black
is squeeze
http://www.blender.org/bf/0001_0014.avi
Quick test with softbody stretch
http://www.blender.org/bf/0001_0100.avi
Based on the difference of the "Orco" (original undeformed coordinate)
and the actual render coordinate, a stress value is computed to make
textures react to stretching or wrinking skin.
The texture coordinate is neutral (0) on relaxed state. -1 is squeezed
to zero, +1 is stretched to infinity.
Note that scaling (object itself or parent) also will result in
stress values.
The reason for the huge commit is a cleanup in allocating memory for
the vertices. These were growing too large with new options, so now it
allocates the optional coordinates dynamically.
Saves about 20 MB memory per 1M vertices already. But best of all is that
I now can add much more fun... so tangents, here we come!
reads from the old mface->edcode flag to set edge drawing.
ALso; added a pointer check in draw_mesh_object(), here the derivedmesh
gives NULL on reading regression file lostride.blend. Zr needs to check!
Radiosity didn't add edges block, on "Replace Meshes". Caused wireframe
not to draw, but also crashes like for join().
Also: added patch that sets the drawflags in edges derived from the
ones set in faces. This ensures the conversion to be done correct.
TODO: this edges call also used in other areas in code, that has to be
catched and done differently.
to get rid of faces with MFace.v3==0
- change all Mesh's to have ->medge now. This is forced by make_edges
on readfile, and in the various exotic important routines, and on
conversion back in python.
- make python NMesh structure always have medges now (needs testing)
- with above two changes it is guarenteed that mf->v3 is never ==0
in main blender code (i.e., all MFace's are actually triangles
or quads) and so I went through and removed all the historic tests
to deal with MFace.v3==0. Equals lots of deleting, I am in heaven!
- removed MEdge edcode flag, no longer needed
- added experimental replacement for edge flag system
Still are some inconsistencies in FACESELECT mode edge drawing to
be ironed out.
NOTE: This commit adds an experimental edge flag calc system, based
on 10-seconds-of-thought algorithm by yours truly. Would appreciate
feedback on how this system works, esp compared to old one and esp
on complex or interesting models.
To Use: New system is enabled by setting G.rt to a value between
1 and 1000 (Value of 0 uses old system). Value 1000 is reserved for
"auto" edge, which is more or less identical to old system but also
makes sure that at least 10% of edges are drawn (solves errors for
super subdivided meshes). Values between 1 and 999 act as percent
(out of 1000) of edges that should be drawn, starting with "most
interesting" edges first. Please try it and comment!
mapping (instead of Edit{Vert,Edge,Face} pointers)
- dropped convertToDispListMeshMapped (whew, glad of it too)
- added DerivedMesh drawMappedFaces function
- dropped EM suffix for DerivedMesh functions, it was neither
particularly correct nor descriptive
- converted test_index_mface to test_index_face that also corrects
MCol and TFace. Good thing we had three versions of this routine,
you never know when one might burn down.
- removed flipnorm_mesh, not used anymore (and was incorrect to
boot)
- Getting face select to work with modifiers turned out to be much
more complicated than expected. Reworked mapping architecture for
modifiers - basically elements in a DispListMesh are now required
to be stored in an order that corresponds exactly to original
ordering. MVert/MEdge/MFace all have a new flag ME_XXX_STEPINDEX
that is set on each element that is set on the first derived element
of each original element. I can't say the code to follow these
requirements for subsurf is particularly transparent, but on the
upside it is a reasonably consistent and simple system that is memory
efficient and allows keeping the DispListMesh structure.
- rewrote mirror modifier to be simpler/conform to new requirements
for mapped DispListMesh structure. This also means that mirror interacts
much better with incremental subsurf calculation (it used to recalc
one entire side on any topology change, now it generally avoids that).
- added EM_{init,free}_index_arrays and EM_get_{vert,edge,face}_for_index
functions to handle mapping indices back into appropriate EditMesh
structures.
- bug fix, make edges didn't recalc object data
- bug fix, initial image assignment to TFace's didn't recalc object data
- new feature, added circle select support for FACESELECT
- bug fix, creating new faces in editmode duplicated the TFACE active
flag - but there should only be one active tface
- bug fix, possible crash when deleting all faces in faceselect mode
on mesh with tfaces...
Still todo: TFace edge drawing is still not always correct in face
mode, in particular with a mirror modifier when mesh has edges (and
no preceeding subsurf). Have not yet decided how to deal with this.
Best solution is probably to do switch to meshes all having MEdge's,
in which case I can get rid of TFace edge flags (and need to recalc
modifiers on tface selection change).
which calculates texspace on demand if need be.
- removed almost all calls to tex_space_mesh
There may be a few corner cases where this goes wrong (meshes with vertex
keys) but these should get ironed out by coming modifier system.
had to add a new feature to the 3d window, to collect "after draw" objects,
which get drawn as last, after a clear of the zbuffer.
Same method can be used for nice OpenGL transparent draw, the system is
ready for it, do that later.
The huge commit is caused by cleaning up globals from struct Global. Many
variables were unused or just not needed anymore. Did that to move the ugly
G.zbuf to where it belongs, in the View3D space struct. :)
