--User changes

-Add support for multiple GPU devices.
  --These options are present in the command line and in Fractorium.
 -Change scheme of specifying devices from platform,device to just total device index.
  --Single number on the command line.
  --Change from combo boxes for device selection to a table of all devices in Fractorium.
 -Temporal samples defaults to 100 instead of 1000 which was needless overkill.

--Bug fixes
 -EmberAnimate, EmberRender, FractoriumSettings, FinalRenderDialog: Fix wrong order of arguments to Clamp() when assigning thread priority.
 -VariationsDC.h: Fix NVidia OpenCL compilation error in DCTriangleVariation.
 -FractoriumXformsColor.cpp: Checking for null pixmap pointer is not enough, must also check if the underlying buffer is null via call to QPixmap::isNull().

--Code changes
 -Ember.h: Add case for FLAME_MOTION_NONE and default in ApplyFlameMotion().
 -EmberMotion.h: Call base constructor.
 -EmberPch.h: #pragma once only on Windows.
 -EmberToXml.h:
  --Handle different types of exceptions.
  --Add default cases to ToString().
 -Isaac.h: Remove unused variable in constructor.
 -Point.h: Call base constructor in Color().
 -Renderer.h/cpp:
  --Add bool to Alloc() to only allocate memory for the histogram. Needed for multi-GPU.
  --Make CoordMap() return a const ref, not a pointer.
 -SheepTools.h:
  --Use 64-bit types like the rest of the code already does.
  --Fix some comment misspellings.
 -Timing.h: Make BeginTime(), EndTime(), ElapsedTime() and Format() be const functions.
 -Utils.h:
  --Add new functions Equal() and Split().
  --Handle more exception types in ReadFile().
  --Get rid of most legacy blending of C and C++ argument parsing.
 -XmlToEmber.h:
  --Get rid of most legacy blending of C and C++ code from flam3.
  --Remove some unused variables.
 -EmberAnimate:
  --Support multi-GPU processing that alternates full frames between devices.
  --Use OpenCLInfo instead of OpenCLWrapper for --openclinfo option.
  --Remove bucketT template parameter, and hard code float in its place.
  --If a render fails, exit since there is no point in continuing an animation with a missing frame.
  --Pass variables to threaded save better, which most likely fixes a very subtle bug that existed before.
  --Remove some unused variables.
 -EmberGenome, EmberRender:
  --Support multi-GPU processing that alternates full frames between devices.
  --Use OpenCLInfo instead of OpenCLWrapper for --openclinfo option.
  --Remove bucketT template parameter, and hard code float in its place.
 -EmberRender:
  --Support multi-GPU processing that alternates full frames between devices.
  --Use OpenCLInfo instead of OpenCLWrapper for --openclinfo option.
  --Remove bucketT template parameter, and hard code float in its place.
  --Only print values when not rendering with OpenCL, since they're always 0 in that case.
 -EmberCLPch.h:
  --#pragma once only on Windows.
  --#include <atomic>.
 -IterOpenCLKernelCreator.h: Add new kernel for summing two histograms. This is needed for multi-GPU.
 -OpenCLWrapper.h:
  --Move all OpenCL info related code into its own class OpenCLInfo.
  --Add members to cache the values of global memory size and max allocation size.
 -RendererCL.h/cpp:
  --Redesign to accomodate multi-GPU.
  --Constructor now takes a vector of devices.
  --Remove DumpErrorReport() function, it's handled in the base.
  --ClearBuffer(), ReadPoints(), WritePoints(), ReadHist() and WriteHist() now optionally take a device index as a parameter.
  --MakeDmap() override and m_DmapCL member removed because it no longer applies since the histogram is always float since the last commit.
  --Add new function SumDeviceHist() to sum histograms from two devices by first copying to a temporary on the host, then a temporary on the device, then summing.
  --m_Calls member removed, as it's now per-device.
  --OpenCLWrapper removed.
  --m_Seeds member is now a vector of vector of seeds, to accomodate a separate and different array of seeds for each device.
  --Added member m_Devices, a vector of unique_ptr of RendererCLDevice.
 -EmberCommon.h
  --Added Devices() function to convert from a vector of device indices to a vector of platform,device indices.
  --Changed CreateRenderer() to accept a vector of devices to create a single RendererCL which will split work across multiple devices.
  --Added CreateRenderers() function to accept a vector of devices to create multiple RendererCL, each which will render on a single device.
  --Add more comments to some existing functions.
 -EmberCommonPch.h: #pragma once only on Windows.
 -EmberOptions.h:
  --Remove --platform option, it's just sequential device number now with the --device option.
  --Make --out be OPT_USE_RENDER instead of OPT_RENDER_ANIM since it's an error condition when animating. It makes no sense to write all frames to a single image.
  --Add Devices() function to parse comma separated --device option string and return a vector of device indices.
  --Make int and uint types be 64-bit, so intmax_t and size_t.
  --Make better use of macros.
 -JpegUtils.h: Make string parameters to WriteJpeg() and WritePng() be const ref.
 -All project files: Turn off buffer security check option in Visual Studio (/Gs-)
 -deployment.pri: Remove the line OTHER_FILES +=, it's pointless and was causing problems.
 -Ember.pro, EmberCL.pro: Add CONFIG += plugin, otherwise it wouldn't link.
 -EmberCL.pro: Add new files for multi-GPU support.
 -build_all.sh: use -j4 and QMAKE=${QMAKE:/usr/bin/qmake}
 -shared_settings.pri:
  -Add version string.
  -Remove old DESTDIR definitions.
  -Add the following lines or else nothing would build:
   CONFIG(release, debug|release) {
    CONFIG += warn_off
    DESTDIR = ../../../Bin/release
   }

   CONFIG(debug, debug|release) {
    DESTDIR = ../../../Bin/debug
   }

   QMAKE_POST_LINK += $$quote(cp --update ../../../Data/flam3-palettes.xml $${DESTDIR}$$escape_expand(\n\t))
   LIBS += -L/usr/lib -lpthread
 -AboutDialog.ui: Another futile attempt to make it look correct on Linux.
 -FinalRenderDialog.h/cpp:
  --Add support for multi-GPU.
  --Change from combo boxes for device selection to a table of all devices.
  --Ensure device selection makes sense.
  --Remove "FinalRender" prefix of various function names, it's implied given the context.
 -FinalRenderEmberController.h/cpp:
  --Add support for multi-GPU.
  --Change m_FinishedImageCount to be atomic.
  --Move CancelRender() from the base to FinalRenderEmberController<T>.
  --Refactor RenderComplete() to omit any progress related functionality or image saving since it can be potentially ran in a thread.
  --Consolidate setting various renderer fields into SyncGuiToRenderer().
 -Fractorium.cpp: Allow for resizing of the options dialog to show the entire device table.
 -FractoriumCommon.h: Add various functions to handle a table showing the available OpenCL devices on the system.
 -FractoriumEmberController.h/cpp: Remove m_FinalImageIndex, it's no longer needed.
 -FractoriumRender.cpp: Scale the interactive sub batch count and quality by the number of devices used.
 -FractoriumSettings.h/cpp:
  --Temporal samples defaults to 100 instead of 1000 which was needless overkill.
  --Add multi-GPU support, remove old device,platform pair.
 -FractoriumToolbar.cpp: Disable OpenCL toolbar button if there are no devices present on the system.
 -FractoriumOptionsDialog.h/cpp:
  --Add support for multi-GPU.
  --Consolidate more assignments in DataToGui().
  --Enable/disable CPU/OpenCL items in response to OpenCL checkbox event.
 -Misc: Convert almost everything to size_t for unsigned, intmax_t for signed.

Source/EmberCommon/EmberCommon.h

@@ -84,7 +84,7 @@ private:
 /// <param name="useDefaults">True to use defaults if they are not present in the file, else false to use invalid values as placeholders to indicate the values were not present. Default: true.</param>
 /// <returns>True if success, else false.</returns>
 template <typename T>
-static bool ParseEmberFile(XmlToEmber<T>& parser, string filename, vector<Ember<T>>& embers, bool useDefaults = true)
+static bool ParseEmberFile(XmlToEmber<T>& parser, const string& filename, vector<Ember<T>>& embers, bool useDefaults = true)
 {
 	if (!parser.Parse(filename.c_str(), embers, useDefaults))
 	{
@@ -138,7 +138,7 @@ static void RgbaToRgb(vector<byte>& rgba, vector<byte>& rgb, size_t width, size_
 	if (rgba.data() != rgb.data())//Only resize the destination buffer if they are different.
 		rgb.resize(width * height * 3);
 
-	for (uint i = 0, j = 0; i < (width * height * 4); i += 4, j += 3)
+	for (size_t i = 0, j = 0; i < (width * height * 4); i += 4, j += 3)
 	{
 		rgb[j]	   = rgba[i];
 		rgb[j + 1] = rgba[i + 1];
@@ -231,35 +231,52 @@ static T NextLowestEvenDiv(T numerator, T denominator)
 	return result;
 }
 
+/// <summary>
+/// Wrapper for converting a vector of absolute device indices to a vector
+/// of platform,device index pairs.
+/// </summary>
+/// <param name="selectedDevices">The vector of absolute device indices to convert</param>
+/// <returns>The converted vector of platform,device index pairs</returns>
+static vector<pair<size_t, size_t>> Devices(const vector<size_t>& selectedDevices)
+{
+	vector<pair<size_t, size_t>> vec;
+	OpenCLInfo& info = OpenCLInfo::Instance();
+	auto& devices = info.DeviceIndices();
+
+	vec.reserve(selectedDevices.size());
+
+	for (size_t i = 0; i < selectedDevices.size(); i++)
+	{
+		auto index = selectedDevices[i];
+
+		if (index < devices.size())
+			vec.push_back(devices[index]);
+	}
+
+	return vec;
+}
+
 /// <summary>
 /// Wrapper for creating a renderer of the specified type.
-/// First template argument expected to be float or double for CPU renderer,
-/// Second argument expected to be float or double for CPU renderer, and only float for OpenCL renderer.
 /// </summary>
 /// <param name="renderType">Type of renderer to create</param>
-/// <param name="platform">The index platform of the platform to use</param>
-/// <param name="device">The index device of the device to use</param>
+/// <param name="devices">The vector of platform/device indices to use</param>
 /// <param name="shared">True if shared with OpenGL, else false.</param>
 /// <param name="texId">The texture ID of the shared OpenGL texture if shared</param>
 /// <param name="errorReport">The error report for holding errors if anything goes wrong</param>
 /// <returns>A pointer to the created renderer if successful, else false.</returns>
-template <typename T, typename bucketT>
-static Renderer<T, bucketT>* CreateRenderer(eRendererType renderType, uint platform, uint device, bool shared, GLuint texId, EmberReport& errorReport)
+template <typename T>
+static Renderer<T, float>* CreateRenderer(eRendererType renderType, const vector<pair<size_t, size_t>>& devices, bool shared, GLuint texId, EmberReport& errorReport)
 {
 	string s;
-	unique_ptr<Renderer<T, bucketT>> renderer;
+	unique_ptr<Renderer<T, float>> renderer;
 
 	try
 	{
-		if (renderType == CPU_RENDERER)
-		{
-			s = "CPU";
-			renderer = unique_ptr<Renderer<T, bucketT>>(new Renderer<T, bucketT>());
-		}
-		else if (renderType == OPENCL_RENDERER)
+		if (renderType == OPENCL_RENDERER && !devices.empty())
 		{
 			s = "OpenCL";
-			renderer = unique_ptr<Renderer<T, bucketT>>(new RendererCL<T, bucketT>(platform, device, shared, texId));
+			renderer = unique_ptr<Renderer<T, float>>(new RendererCL<T, float>(devices, shared, texId));
 
 			if (!renderer.get() || !renderer->Ok())
 			{
@@ -267,9 +284,18 @@ static Renderer<T, bucketT>* CreateRenderer(eRendererType renderType, uint platf
 					errorReport.AddToReport(renderer->ErrorReport());
 
 				errorReport.AddToReport("Error initializing OpenCL renderer, using CPU renderer instead.");
-				renderer = unique_ptr<Renderer<T, bucketT>>(new Renderer<T, bucketT>());
+				renderer = unique_ptr<Renderer<T, float>>(new Renderer<T, float>());
 			}
 		}
+		else
+		{
+			s = "CPU";
+			renderer = unique_ptr<Renderer<T, float>>(new Renderer<T, float>());
+		}
+	}
+	catch (const std::exception& e)
+	{
+		errorReport.AddToReport("Error creating " + s + " renderer: " + e.what() + "\n");
 	}
 	catch (...)
 	{
@@ -279,6 +305,100 @@ static Renderer<T, bucketT>* CreateRenderer(eRendererType renderType, uint platf
 	return renderer.release();
 }
 
+/// <summary>
+/// Wrapper for creating a vector of renderers of the specified type for each passed in device.
+/// If shared is true, only the first renderer will be shared with OpenGL.
+/// Although a fallback GPU renderer will be created if a failure occurs, it doesn't really
+/// make sense since the concept of devices only applies to OpenCL renderers.
+/// </summary>
+/// <param name="renderType">Type of renderer to create</param>
+/// <param name="devices">The vector of platform/device indices to use</param>
+/// <param name="shared">True if shared with OpenGL, else false.</param>
+/// <param name="texId">The texture ID of the shared OpenGL texture if shared</param>
+/// <param name="errorReport">The error report for holding errors if anything goes wrong</param>
+/// <returns>The vector of created renderers if successful, else false.</returns>
+template <typename T>
+static vector<unique_ptr<Renderer<T, float>>> CreateRenderers(eRendererType renderType, const vector<pair<size_t, size_t>>& devices, bool shared, GLuint texId, EmberReport& errorReport)
+{
+	string s;
+	vector<unique_ptr<Renderer<T, float>>> v;
+
+	try
+	{
+		if (renderType == OPENCL_RENDERER && !devices.empty())
+		{
+			s = "OpenCL";
+			v.reserve(devices.size());
+
+			for (size_t i = 0; i < devices.size(); i++)
+			{
+				vector<pair<size_t, size_t>> tempDevices{ devices[i] };
+				auto renderer = unique_ptr<Renderer<T, float>>(new RendererCL<T, float>(tempDevices, !i ? shared : false, texId));
+
+				if (!renderer.get() || !renderer->Ok())
+				{
+					ostringstream os;
+
+					if (renderer.get())
+						errorReport.AddToReport(renderer->ErrorReport());
+
+					os << "Error initializing OpenCL renderer for platform " << devices[i].first << ", " << devices[i].second;
+					errorReport.AddToReport(os.str());
+				}
+				else
+					v.push_back(std::move(renderer));
+			}
+		}
+		else
+		{
+			s = "CPU";
+			v.push_back(std::move(unique_ptr<Renderer<T, float>>(::CreateRenderer<T>(CPU_RENDERER, devices, shared, texId, errorReport))));
+		}
+	}
+	catch (const std::exception& e)
+	{
+		errorReport.AddToReport("Error creating " + s + " renderer: " + e.what() + "\n");
+	}
+	catch (...)
+	{
+		errorReport.AddToReport("Error creating " + s + " renderer.\n");
+	}
+
+	if (v.empty() && s != "CPU")//OpenCL creation failed and CPU creation has not been attempted, so just create one CPU renderer and place it in the vector.
+	{
+		try
+		{
+			s = "CPU";
+			v.push_back(std::move(unique_ptr<Renderer<T, float>>(::CreateRenderer<T>(CPU_RENDERER, devices, shared, texId, errorReport))));
+		}
+		catch (const std::exception& e)
+		{
+			errorReport.AddToReport("Error creating fallback" + s + " renderer: " + e.what() + "\n");
+		}
+		catch (...)
+		{
+			errorReport.AddToReport("Error creating fallback " + s + " renderer.\n");
+		}
+	}
+
+	return v;
+}
+
+/// <summary>
+/// Perform a render which allows for using strips or not.
+/// If an error occurs while rendering any strip, the rendering process stops.
+/// </summary>
+/// <param name="renderer">The renderer to use</param>
+/// <param name="ember">The ember to render</param>
+/// <param name="finalImage">The vector to place the final output in</param>
+/// <param name="time">The time position to use, only valid for animation</param>
+/// <param name="strips">The number of strips to use. This must be validated before calling this function.</param>
+/// <param name="yAxisUp">True to flip the Y axis, else false.</param>
+/// <param name="perStripStart">Function called before the start of the rendering of each strip</param>
+/// <param name="perStripFinish">Function called after the end of the rendering of each strip</param>
+/// <param name="perStripError">Function called if there is an error rendering a strip</param>
+/// <param name="allStripsFinished">Function called when all strips successfully finish rendering</param>
+/// <returns>True if all rendering was successful, else false.</returns>
 template <typename T>
 static bool StripsRender(RendererBase* renderer, Ember<T>& ember, vector<byte>& finalImage, double time, size_t strips, bool yAxisUp,
 	std::function<void(size_t strip)> perStripStart,
@@ -354,6 +474,17 @@ static bool StripsRender(RendererBase* renderer, Ember<T>& ember, vector<byte>&
 	return success;
 }
 
+/// <summary>
+/// Verify that the specified number of strips is valid for the given height.
+/// The passed in error functions will be called if the number of strips needs
+/// to be modified for the given height.
+/// </summary>
+/// <param name="height">The height in pixels of the image to be rendered</param>
+/// <param name="strips">The number of strips to split the render into</param>
+/// <param name="stripError1">Function called if the number of strips exceeds the height of the image</param>
+/// <param name="stripError2">Function called if the number of strips does not divide evently into the height of the image</param>
+/// <param name="stripError3">Called if for any reason the number of strips used will differ from the value passed in</param>
+/// <returns>The actual number of strips that will be used</returns>
 static size_t VerifyStrips(size_t height, size_t strips,
 	std::function<void(const string& s)> stripError1,
 	std::function<void(const string& s)> stripError2,

Source/EmberCommon/EmberCommonPch.h

@@ -1,4 +1,6 @@
-#pragma once
+#ifdef WIN32
+    #pragma once
+#endif
 
 /// <summary>
 /// Precompiled header file. Place all system includes here with appropriate #defines for different operating systems and compilers.

Source/EmberCommon/EmberOptions.h

@@ -56,9 +56,7 @@ enum eOptionIDs
 	OPT_DUMP_KERNEL,
 
 	//Value args.
-	OPT_OPENCL_PLATFORM,//Int value args.
-	OPT_OPENCL_DEVICE,
-	OPT_SEED,
+	OPT_SEED,//Int value args.
 	OPT_NTHREADS,
 	OPT_STRIPS,
 	OPT_SUPERSAMPLE,
@@ -94,7 +92,8 @@ enum eOptionIDs
 	OPT_USEMEM,
 	OPT_LOOPS,
 
-	OPT_ISAAC_SEED,//String value args.
+	OPT_OPENCL_DEVICE,//String value args.
+	OPT_ISAAC_SEED,
 	OPT_IN,
 	OPT_OUT,
 	OPT_PREFIX,
@@ -158,7 +157,7 @@ public:
 	/// <param name="defaultVal">The default value to use the option was not given on the command line</param>
 	/// <param name="argType">The format the argument should be given in</param>
 	/// <param name="docString">The documentation string describing what the argument means</param>
-	EmberOptionEntry(eOptionUse optUsage, eOptionIDs optId, const CharT* arg, T defaultVal, ESOArgType argType, string docString)
+	EmberOptionEntry(eOptionUse optUsage, eOptionIDs optId, const CharT* arg, T defaultVal, ESOArgType argType, const string& docString)
 	{
 		m_OptionUse = optUsage;
 		m_Option.nId = int(optId);
@@ -235,25 +234,25 @@ private:
 		break
 
 //Int.
-#define Eoi EmberOptionEntry<int>
+#define Eoi EmberOptionEntry<intmax_t>
 #define INITINTOPTION(member, option) \
 		member = option; \
 		m_IntArgs.push_back(&member)
 
 #define PARSEINTOPTION(opt, member) \
 	case (opt): \
-		sscanf_s(args.OptionArg(), "%d", &member.m_Val); \
+		sscanf_s(args.OptionArg(), "%ld", &member.m_Val); \
 		break
 
 //Uint.
-#define Eou EmberOptionEntry<uint>
+#define Eou EmberOptionEntry<size_t>
 #define INITUINTOPTION(member, option) \
 		member = option; \
 		m_UintArgs.push_back(&member)
 
 #define PARSEUINTOPTION(opt, member) \
 	case (opt): \
-		sscanf_s(args.OptionArg(), "%u", &member.m_Val); \
+		sscanf_s(args.OptionArg(), "%lu", &member.m_Val); \
 		break
 
 //Double.
@@ -318,7 +317,7 @@ public:
 		INITBOOLOPTION(JpegComments,   Eob(OPT_RENDER_ANIM, OPT_JPEG_COMMENTS,	  _T("--enable_jpeg_comments"), true,				  SO_OPT,	  "\t--enable_jpeg_comments   Enables comments in the jpeg header [default: true].\n"));
 		INITBOOLOPTION(PngComments,    Eob(OPT_RENDER_ANIM, OPT_PNG_COMMENTS,	  _T("--enable_png_comments"),  true,				  SO_OPT,	  "\t--enable_png_comments    Enables comments in the png header [default: true].\n"));
 		INITBOOLOPTION(WriteGenome,    Eob(OPT_USE_ANIMATE, OPT_WRITE_GENOME,     _T("--write_genome"),         false,                SO_NONE,    "\t--write_genome           Write out flame associated with center of motion blur window [default: false].\n"));
-		INITBOOLOPTION(ThreadedWrite,  Eob(OPT_RENDER_ANIM, OPT_THREADED_WRITE,	  _T("--threaded_write"),		true,				  SO_OPT,	  "\t--threaded_write         Use a separate thread to write images to disk. This doubles the memory required for the final output buffer. [default: true].\n"));
+		INITBOOLOPTION(ThreadedWrite,  Eob(OPT_RENDER_ANIM, OPT_THREADED_WRITE,	  _T("--threaded_write"),		true,				  SO_OPT,	  "\t--threaded_write         Use a separate thread to write images to disk. This gives better performance, but doubles the memory required for the final output buffer. [default: true].\n"));
 		INITBOOLOPTION(Enclosed,	   Eob(OPT_USE_GENOME,  OPT_ENCLOSED,		  _T("--enclosed"),				true,				  SO_OPT,	  "\t--enclosed               Use enclosing XML tags [default: true].\n"));
 		INITBOOLOPTION(NoEdits,        Eob(OPT_USE_GENOME,  OPT_NO_EDITS,         _T("--noedits"),              false,                SO_NONE,    "\t--noedits                Exclude edit tags when writing Xml [default: false].\n"));
 		INITBOOLOPTION(UnsmoothEdge,   Eob(OPT_USE_GENOME,  OPT_UNSMOOTH_EDGE,    _T("--unsmoother"),           false,                SO_NONE,    "\t--unsmoother             Do not use smooth blending for sheep edges [default: false].\n"));
@@ -336,8 +335,6 @@ public:
 #endif
 
 		//Uint.
-		INITUINTOPTION(Platform,       Eou(OPT_USE_ALL,     OPT_OPENCL_PLATFORM,  _T("--platform"),             0,                    SO_REQ_SEP, "\t--platform               The OpenCL platform index to use [default: 0].\n"));
-		INITUINTOPTION(Device,         Eou(OPT_USE_ALL,     OPT_OPENCL_DEVICE,    _T("--device"),               0,                    SO_REQ_SEP, "\t--device                 The OpenCL device index within the specified platform to use [default: 0].\n"));
 		INITUINTOPTION(Seed,           Eou(OPT_USE_ALL,     OPT_SEED,             _T("--seed"),                 0,                    SO_REQ_SEP, "\t--seed=<val>             Integer seed to use for the random number generator [default: random].\n"));
 		INITUINTOPTION(ThreadCount,    Eou(OPT_USE_ALL,     OPT_NTHREADS,         _T("--nthreads"),             0,                    SO_REQ_SEP, "\t--nthreads=<val>         The number of threads to use [default: use all available cores].\n"));
 		INITUINTOPTION(Strips,		   Eou(OPT_USE_RENDER,  OPT_STRIPS,           _T("--nstrips"),              1,                    SO_REQ_SEP, "\t--nstrips=<val>          The number of fractions to split a single render frame into. Useful for print size renders or low memory systems [default: 1].\n"));
@@ -378,9 +375,10 @@ public:
 		INITDOUBLEOPTION(Loops,        Eod(OPT_USE_GENOME,  OPT_LOOPS,            _T("--loops"),                1.0,                  SO_REQ_SEP, "\t--loops=<val>            Number of times to rotate each control point in sequence [default: 1].\n"));
 
 		//String.
+		INITSTRINGOPTION(Device,	   Eos(OPT_USE_ALL,		OPT_OPENCL_DEVICE,	  _T("--device"),				"0",				  SO_REQ_SEP, "\t--device                 The comma-separated OpenCL device indices to use. Single device: 0 Multi device: 0,1,3,4 [default: 0].\n"));
 		INITSTRINGOPTION(IsaacSeed,    Eos(OPT_USE_ALL,     OPT_ISAAC_SEED,       _T("--isaac_seed"),           "",                   SO_REQ_SEP, "\t--isaac_seed=<val>       Character-based seed for the random number generator [default: random].\n"));
 		INITSTRINGOPTION(Input,        Eos(OPT_RENDER_ANIM, OPT_IN,               _T("--in"),                   "",                   SO_REQ_SEP, "\t--in=<val>               Name of the input file.\n"));
-		INITSTRINGOPTION(Out,          Eos(OPT_RENDER_ANIM, OPT_OUT,              _T("--out"),                  "",                   SO_REQ_SEP, "\t--out=<val>              Name of a single output file. Not recommended when rendering more than one image.\n"));
+		INITSTRINGOPTION(Out,          Eos(OPT_USE_RENDER,	OPT_OUT,              _T("--out"),                  "",                   SO_REQ_SEP, "\t--out=<val>              Name of a single output file. Not recommended when rendering more than one image.\n"));
 		INITSTRINGOPTION(Prefix,       Eos(OPT_RENDER_ANIM, OPT_PREFIX,           _T("--prefix"),               "",                   SO_REQ_SEP, "\t--prefix=<val>           Prefix to prepend to all output files.\n"));
 		INITSTRINGOPTION(Suffix,       Eos(OPT_RENDER_ANIM, OPT_SUFFIX,           _T("--suffix"),               "",                   SO_REQ_SEP, "\t--suffix=<val>           Suffix to append to all output files.\n"));
 		INITSTRINGOPTION(Format,       Eos(OPT_RENDER_ANIM, OPT_FORMAT,           _T("--format"),               "png",                SO_REQ_SEP, "\t--format=<val>           Format of the output file. Valid values are: bmp, jpg, png, ppm [default: jpg].\n"));
@@ -470,9 +468,7 @@ public:
 					PARSEINTOPTION(OPT_SHEEP_GEN, SheepGen);
 					PARSEINTOPTION(OPT_SHEEP_ID, SheepId);
 					PARSEINTOPTION(OPT_PRIORITY, Priority);
-					PARSEUINTOPTION(OPT_OPENCL_PLATFORM, Platform);//uint args.
-					PARSEUINTOPTION(OPT_OPENCL_DEVICE, Device);
-					PARSEUINTOPTION(OPT_SEED, Seed);
+					PARSEUINTOPTION(OPT_SEED, Seed);//uint args.
 					PARSEUINTOPTION(OPT_NTHREADS, ThreadCount);
 					PARSEUINTOPTION(OPT_STRIPS, Strips);
 					PARSEUINTOPTION(OPT_SUPERSAMPLE, Supersample);
@@ -504,7 +500,8 @@ public:
 					PARSEDOUBLEOPTION(OPT_USEMEM, UseMem);
 					PARSEDOUBLEOPTION(OPT_LOOPS, Loops);
 
-					PARSESTRINGOPTION(OPT_ISAAC_SEED, IsaacSeed);//String args.
+					PARSESTRINGOPTION(OPT_OPENCL_DEVICE, Device);//String args.
+					PARSESTRINGOPTION(OPT_ISAAC_SEED, IsaacSeed);
 					PARSESTRINGOPTION(OPT_IN, Input);
 					PARSESTRINGOPTION(OPT_OUT, Out);
 					PARSESTRINGOPTION(OPT_PREFIX, Prefix);
@@ -545,9 +542,36 @@ public:
 			}
 		}
 
+		auto strings = Split(Device(), ',');
+
+		if (!strings.empty())
+		{
+			for (auto& s : strings)
+			{
+				size_t device = 0;
+				istringstream istr(s);
+
+				istr >> device;
+
+				if (!istr.bad() && !istr.fail())
+					m_Devices.push_back(device);
+				else
+					cout << "Failed to parse device index " << s;
+			}
+		}
+
 		return false;
 	}
 
+	/// <summary>
+	/// Return a const ref to m_Devices.
+	/// </summary>
+	/// <returns>A const ref to the vector of absolute device indices to be used</returns>
+	const vector<size_t>& Devices()
+	{
+		return m_Devices;
+	}
+
 	/// <summary>
 	/// Return a vector of all available options for the specified program.
 	/// </summary>
@@ -656,103 +680,103 @@ public:
 	//Break from the usual m_* notation for members here because
 	//each of these is a functor, so it looks nicer and is less typing
 	//to just say opt.Member().
-	EmberOptionEntry<bool> Help;//Diagnostic bool.
-	EmberOptionEntry<bool> Version;
-	EmberOptionEntry<bool> Verbose;
-	EmberOptionEntry<bool> Debug;
-	EmberOptionEntry<bool> DumpArgs;
-	EmberOptionEntry<bool> DoProgress;
-	EmberOptionEntry<bool> OpenCLInfo;
+	Eob Help;//Diagnostic bool.
+	Eob Version;
+	Eob Verbose;
+	Eob Debug;
+	Eob DumpArgs;
+	Eob DoProgress;
+	Eob OpenCLInfo;
 
-	EmberOptionEntry<bool> EmberCL;//Value bool.
-	EmberOptionEntry<bool> EarlyClip;
-	EmberOptionEntry<bool> YAxisUp;
-	EmberOptionEntry<bool> Transparency;
-	EmberOptionEntry<bool> NameEnable;
-	EmberOptionEntry<bool> IntPalette;
-	EmberOptionEntry<bool> HexPalette;
-	EmberOptionEntry<bool> InsertPalette;
-	EmberOptionEntry<bool> JpegComments;
-	EmberOptionEntry<bool> PngComments;
-	EmberOptionEntry<bool> WriteGenome;
-	EmberOptionEntry<bool> ThreadedWrite;
-	EmberOptionEntry<bool> Enclosed;
-	EmberOptionEntry<bool> NoEdits;
-	EmberOptionEntry<bool> UnsmoothEdge;
-	EmberOptionEntry<bool> LockAccum;
-	EmberOptionEntry<bool> DumpKernel;
+	Eob EmberCL;//Value bool.
+	Eob EarlyClip;
+	Eob YAxisUp;
+	Eob Transparency;
+	Eob NameEnable;
+	Eob IntPalette;
+	Eob HexPalette;
+	Eob InsertPalette;
+	Eob JpegComments;
+	Eob PngComments;
+	Eob WriteGenome;
+	Eob ThreadedWrite;
+	Eob Enclosed;
+	Eob NoEdits;
+	Eob UnsmoothEdge;
+	Eob LockAccum;
+	Eob DumpKernel;
 
-	EmberOptionEntry<int> Symmetry;//Value int.
-	EmberOptionEntry<int> SheepGen;
-	EmberOptionEntry<int> SheepId;
-	EmberOptionEntry<int> Priority;
-	EmberOptionEntry<uint> Platform;//Value uint.
-	EmberOptionEntry<uint> Device;
-	EmberOptionEntry<uint> Seed;
-	EmberOptionEntry<uint> ThreadCount;
-	EmberOptionEntry<uint> Strips;
-	EmberOptionEntry<uint> Supersample;
-	EmberOptionEntry<uint> BitsPerChannel;
-	EmberOptionEntry<uint> SubBatchSize;
-	EmberOptionEntry<uint> Bits;
-	EmberOptionEntry<uint> PrintEditDepth;
-	EmberOptionEntry<uint> JpegQuality;
-	EmberOptionEntry<uint> FirstFrame;
-	EmberOptionEntry<uint> LastFrame;
-	EmberOptionEntry<uint> Frame;
-	EmberOptionEntry<uint> Time;
-	EmberOptionEntry<uint> Dtime;
-	EmberOptionEntry<uint> Frames;
-	EmberOptionEntry<uint> Repeat;
-	EmberOptionEntry<uint> Tries;
-	EmberOptionEntry<uint> MaxXforms;
+	Eoi Symmetry;//Value int.
+	Eoi SheepGen;
+	Eoi SheepId;
+	Eoi Priority;
+	Eou Seed;//Value uint.
+	Eou ThreadCount;
+	Eou Strips;
+	Eou Supersample;
+	Eou BitsPerChannel;
+	Eou SubBatchSize;
+	Eou Bits;
+	Eou PrintEditDepth;
+	Eou JpegQuality;
+	Eou FirstFrame;
+	Eou LastFrame;
+	Eou Frame;
+	Eou Time;
+	Eou Dtime;
+	Eou Frames;
+	Eou Repeat;
+	Eou Tries;
+	Eou MaxXforms;
 
-	EmberOptionEntry<double> SizeScale;//Value double.
-	EmberOptionEntry<double> QualityScale;
-	EmberOptionEntry<double> AspectRatio;
-	EmberOptionEntry<double> Stagger;
-	EmberOptionEntry<double> AvgThresh;
-	EmberOptionEntry<double> BlackThresh;
-	EmberOptionEntry<double> WhiteLimit;
-	EmberOptionEntry<double> Speed;
-	EmberOptionEntry<double> OffsetX;
-	EmberOptionEntry<double> OffsetY;
-	EmberOptionEntry<double> UseMem;
-	EmberOptionEntry<double> Loops;
+	Eod SizeScale;//Value double.
+	Eod QualityScale;
+	Eod AspectRatio;
+	Eod Stagger;
+	Eod AvgThresh;
+	Eod BlackThresh;
+	Eod WhiteLimit;
+	Eod Speed;
+	Eod OffsetX;
+	Eod OffsetY;
+	Eod UseMem;
+	Eod Loops;
 
-	EmberOptionEntry<string> IsaacSeed;//Value string.
-	EmberOptionEntry<string> Input;
-	EmberOptionEntry<string> Out;
-	EmberOptionEntry<string> Prefix;
-	EmberOptionEntry<string> Suffix;
-	EmberOptionEntry<string> Format;
-	EmberOptionEntry<string> PalettePath;
-	//EmberOptionEntry<string> PaletteImage;
-	EmberOptionEntry<string> Id;
-	EmberOptionEntry<string> Url;
-	EmberOptionEntry<string> Nick;
-	EmberOptionEntry<string> Comment;
-	EmberOptionEntry<string> TemplateFile;
-	EmberOptionEntry<string> Clone;
-	EmberOptionEntry<string> CloneAll;
-	EmberOptionEntry<string> CloneAction;
-	EmberOptionEntry<string> Animate;
-	EmberOptionEntry<string> Mutate;
-	EmberOptionEntry<string> Cross0;
-	EmberOptionEntry<string> Cross1;
-	EmberOptionEntry<string> Method;
-	EmberOptionEntry<string> Inter;
-	EmberOptionEntry<string> Rotate;
-	EmberOptionEntry<string> Strip;
-	EmberOptionEntry<string> Sequence;
-	EmberOptionEntry<string> UseVars;
-	EmberOptionEntry<string> DontUseVars;
-	EmberOptionEntry<string> Extras;
+	Eos Device;//Value string.
+	Eos IsaacSeed;
+	Eos Input;
+	Eos Out;
+	Eos Prefix;
+	Eos Suffix;
+	Eos Format;
+	Eos PalettePath;
+	//Eos PaletteImage;
+	Eos Id;
+	Eos Url;
+	Eos Nick;
+	Eos Comment;
+	Eos TemplateFile;
+	Eos Clone;
+	Eos CloneAll;
+	Eos CloneAction;
+	Eos Animate;
+	Eos Mutate;
+	Eos Cross0;
+	Eos Cross1;
+	Eos Method;
+	Eos Inter;
+	Eos Rotate;
+	Eos Strip;
+	Eos Sequence;
+	Eos UseVars;
+	Eos DontUseVars;
+	Eos Extras;
 
 private:
-	vector<EmberOptionEntry<bool>*> m_BoolArgs;
-	vector<EmberOptionEntry<int>*> m_IntArgs;
-	vector<EmberOptionEntry<uint>*> m_UintArgs;
-	vector<EmberOptionEntry<double>*> m_DoubleArgs;
-	vector<EmberOptionEntry<string>*> m_StringArgs;
+	vector<size_t> m_Devices;
+	vector<Eob*> m_BoolArgs;
+	vector<Eoi*> m_IntArgs;
+	vector<Eou*> m_UintArgs;
+	vector<Eod*> m_DoubleArgs;
+	vector<Eos*> m_StringArgs;
 };

Source/EmberCommon/JpegUtils.h

@@ -44,7 +44,7 @@ static bool WritePpm(const char* filename, byte* image, size_t width, size_t hei
 /// <param name="url">Url of the author</param>
 /// <param name="nick">Nickname of the author</param>
 /// <returns>True if success, else false</returns>
-static bool WriteJpeg(const char* filename, byte* image, size_t width, size_t height, int quality, bool enableComments, EmberImageComments& comments, string id, string url, string nick)
+static bool WriteJpeg(const char* filename, byte* image, size_t width, size_t height, int quality, bool enableComments, const EmberImageComments& comments, const string& id, const string& url, const string& nick)
 {
 	bool b = false;
 	FILE* file;
@@ -135,7 +135,7 @@ static bool WriteJpeg(const char* filename, byte* image, size_t width, size_t he
 /// <param name="url">Url of the author</param>
 /// <param name="nick">Nickname of the author</param>
 /// <returns>True if success, else false</returns>
-static bool WritePng(const char* filename, byte* image, size_t width, size_t height, size_t bytesPerChannel, bool enableComments, EmberImageComments& comments, string id, string url, string nick)
+static bool WritePng(const char* filename, byte* image, size_t width, size_t height, size_t bytesPerChannel, bool enableComments, const EmberImageComments& comments, const string& id, const string& url, const string& nick)
 {
 	bool b = false;
 	FILE* file;

Source/EmberCommon/SimpleGlob.h

@@ -601,7 +601,7 @@ private:
 	CSimpleGlobTempl(const CSimpleGlobTempl &); // disabled
 	CSimpleGlobTempl & operator=(const CSimpleGlobTempl &); // disabled
 
-	/*! @brief The argv array has it's members stored as either an offset into
+	/*! @brief The argv array has its members stored as either an offset into
 		the string buffer, or as pointers to their string in the buffer. The
 		offsets are used because if the string buffer is dynamically resized,
 		all pointers into that buffer would become invalid.