--User changes

-Add support for multiple GPU devices.
  --These options are present in the command line and in Fractorium.
 -Change scheme of specifying devices from platform,device to just total device index.
  --Single number on the command line.
  --Change from combo boxes for device selection to a table of all devices in Fractorium.
 -Temporal samples defaults to 100 instead of 1000 which was needless overkill.

--Bug fixes
 -EmberAnimate, EmberRender, FractoriumSettings, FinalRenderDialog: Fix wrong order of arguments to Clamp() when assigning thread priority.
 -VariationsDC.h: Fix NVidia OpenCL compilation error in DCTriangleVariation.
 -FractoriumXformsColor.cpp: Checking for null pixmap pointer is not enough, must also check if the underlying buffer is null via call to QPixmap::isNull().

--Code changes
 -Ember.h: Add case for FLAME_MOTION_NONE and default in ApplyFlameMotion().
 -EmberMotion.h: Call base constructor.
 -EmberPch.h: #pragma once only on Windows.
 -EmberToXml.h:
  --Handle different types of exceptions.
  --Add default cases to ToString().
 -Isaac.h: Remove unused variable in constructor.
 -Point.h: Call base constructor in Color().
 -Renderer.h/cpp:
  --Add bool to Alloc() to only allocate memory for the histogram. Needed for multi-GPU.
  --Make CoordMap() return a const ref, not a pointer.
 -SheepTools.h:
  --Use 64-bit types like the rest of the code already does.
  --Fix some comment misspellings.
 -Timing.h: Make BeginTime(), EndTime(), ElapsedTime() and Format() be const functions.
 -Utils.h:
  --Add new functions Equal() and Split().
  --Handle more exception types in ReadFile().
  --Get rid of most legacy blending of C and C++ argument parsing.
 -XmlToEmber.h:
  --Get rid of most legacy blending of C and C++ code from flam3.
  --Remove some unused variables.
 -EmberAnimate:
  --Support multi-GPU processing that alternates full frames between devices.
  --Use OpenCLInfo instead of OpenCLWrapper for --openclinfo option.
  --Remove bucketT template parameter, and hard code float in its place.
  --If a render fails, exit since there is no point in continuing an animation with a missing frame.
  --Pass variables to threaded save better, which most likely fixes a very subtle bug that existed before.
  --Remove some unused variables.
 -EmberGenome, EmberRender:
  --Support multi-GPU processing that alternates full frames between devices.
  --Use OpenCLInfo instead of OpenCLWrapper for --openclinfo option.
  --Remove bucketT template parameter, and hard code float in its place.
 -EmberRender:
  --Support multi-GPU processing that alternates full frames between devices.
  --Use OpenCLInfo instead of OpenCLWrapper for --openclinfo option.
  --Remove bucketT template parameter, and hard code float in its place.
  --Only print values when not rendering with OpenCL, since they're always 0 in that case.
 -EmberCLPch.h:
  --#pragma once only on Windows.
  --#include <atomic>.
 -IterOpenCLKernelCreator.h: Add new kernel for summing two histograms. This is needed for multi-GPU.
 -OpenCLWrapper.h:
  --Move all OpenCL info related code into its own class OpenCLInfo.
  --Add members to cache the values of global memory size and max allocation size.
 -RendererCL.h/cpp:
  --Redesign to accomodate multi-GPU.
  --Constructor now takes a vector of devices.
  --Remove DumpErrorReport() function, it's handled in the base.
  --ClearBuffer(), ReadPoints(), WritePoints(), ReadHist() and WriteHist() now optionally take a device index as a parameter.
  --MakeDmap() override and m_DmapCL member removed because it no longer applies since the histogram is always float since the last commit.
  --Add new function SumDeviceHist() to sum histograms from two devices by first copying to a temporary on the host, then a temporary on the device, then summing.
  --m_Calls member removed, as it's now per-device.
  --OpenCLWrapper removed.
  --m_Seeds member is now a vector of vector of seeds, to accomodate a separate and different array of seeds for each device.
  --Added member m_Devices, a vector of unique_ptr of RendererCLDevice.
 -EmberCommon.h
  --Added Devices() function to convert from a vector of device indices to a vector of platform,device indices.
  --Changed CreateRenderer() to accept a vector of devices to create a single RendererCL which will split work across multiple devices.
  --Added CreateRenderers() function to accept a vector of devices to create multiple RendererCL, each which will render on a single device.
  --Add more comments to some existing functions.
 -EmberCommonPch.h: #pragma once only on Windows.
 -EmberOptions.h:
  --Remove --platform option, it's just sequential device number now with the --device option.
  --Make --out be OPT_USE_RENDER instead of OPT_RENDER_ANIM since it's an error condition when animating. It makes no sense to write all frames to a single image.
  --Add Devices() function to parse comma separated --device option string and return a vector of device indices.
  --Make int and uint types be 64-bit, so intmax_t and size_t.
  --Make better use of macros.
 -JpegUtils.h: Make string parameters to WriteJpeg() and WritePng() be const ref.
 -All project files: Turn off buffer security check option in Visual Studio (/Gs-)
 -deployment.pri: Remove the line OTHER_FILES +=, it's pointless and was causing problems.
 -Ember.pro, EmberCL.pro: Add CONFIG += plugin, otherwise it wouldn't link.
 -EmberCL.pro: Add new files for multi-GPU support.
 -build_all.sh: use -j4 and QMAKE=${QMAKE:/usr/bin/qmake}
 -shared_settings.pri:
  -Add version string.
  -Remove old DESTDIR definitions.
  -Add the following lines or else nothing would build:
   CONFIG(release, debug|release) {
    CONFIG += warn_off
    DESTDIR = ../../../Bin/release
   }

   CONFIG(debug, debug|release) {
    DESTDIR = ../../../Bin/debug
   }

   QMAKE_POST_LINK += $$quote(cp --update ../../../Data/flam3-palettes.xml $${DESTDIR}$$escape_expand(\n\t))
   LIBS += -L/usr/lib -lpthread
 -AboutDialog.ui: Another futile attempt to make it look correct on Linux.
 -FinalRenderDialog.h/cpp:
  --Add support for multi-GPU.
  --Change from combo boxes for device selection to a table of all devices.
  --Ensure device selection makes sense.
  --Remove "FinalRender" prefix of various function names, it's implied given the context.
 -FinalRenderEmberController.h/cpp:
  --Add support for multi-GPU.
  --Change m_FinishedImageCount to be atomic.
  --Move CancelRender() from the base to FinalRenderEmberController<T>.
  --Refactor RenderComplete() to omit any progress related functionality or image saving since it can be potentially ran in a thread.
  --Consolidate setting various renderer fields into SyncGuiToRenderer().
 -Fractorium.cpp: Allow for resizing of the options dialog to show the entire device table.
 -FractoriumCommon.h: Add various functions to handle a table showing the available OpenCL devices on the system.
 -FractoriumEmberController.h/cpp: Remove m_FinalImageIndex, it's no longer needed.
 -FractoriumRender.cpp: Scale the interactive sub batch count and quality by the number of devices used.
 -FractoriumSettings.h/cpp:
  --Temporal samples defaults to 100 instead of 1000 which was needless overkill.
  --Add multi-GPU support, remove old device,platform pair.
 -FractoriumToolbar.cpp: Disable OpenCL toolbar button if there are no devices present on the system.
 -FractoriumOptionsDialog.h/cpp:
  --Add support for multi-GPU.
  --Consolidate more assignments in DataToGui().
  --Enable/disable CPU/OpenCL items in response to OpenCL checkbox event.
 -Misc: Convert almost everything to size_t for unsigned, intmax_t for signed.

Source/EmberCommon/EmberCommon.h

@@ -84,7 +84,7 @@ private:
 /// <param name="useDefaults">True to use defaults if they are not present in the file, else false to use invalid values as placeholders to indicate the values were not present. Default: true.</param>
 /// <returns>True if success, else false.</returns>
 template <typename T>
-static bool ParseEmberFile(XmlToEmber<T>& parser, string filename, vector<Ember<T>>& embers, bool useDefaults = true)
+static bool ParseEmberFile(XmlToEmber<T>& parser, const string& filename, vector<Ember<T>>& embers, bool useDefaults = true)
 {
 	if (!parser.Parse(filename.c_str(), embers, useDefaults))
 	{
@@ -138,7 +138,7 @@ static void RgbaToRgb(vector<byte>& rgba, vector<byte>& rgb, size_t width, size_
 	if (rgba.data() != rgb.data())//Only resize the destination buffer if they are different.
 		rgb.resize(width * height * 3);
 
-	for (uint i = 0, j = 0; i < (width * height * 4); i += 4, j += 3)
+	for (size_t i = 0, j = 0; i < (width * height * 4); i += 4, j += 3)
 	{
 		rgb[j]	   = rgba[i];
 		rgb[j + 1] = rgba[i + 1];
@@ -231,35 +231,52 @@ static T NextLowestEvenDiv(T numerator, T denominator)
 	return result;
 }
 
+/// <summary>
+/// Wrapper for converting a vector of absolute device indices to a vector
+/// of platform,device index pairs.
+/// </summary>
+/// <param name="selectedDevices">The vector of absolute device indices to convert</param>
+/// <returns>The converted vector of platform,device index pairs</returns>
+static vector<pair<size_t, size_t>> Devices(const vector<size_t>& selectedDevices)
+{
+	vector<pair<size_t, size_t>> vec;
+	OpenCLInfo& info = OpenCLInfo::Instance();
+	auto& devices = info.DeviceIndices();
+
+	vec.reserve(selectedDevices.size());
+
+	for (size_t i = 0; i < selectedDevices.size(); i++)
+	{
+		auto index = selectedDevices[i];
+
+		if (index < devices.size())
+			vec.push_back(devices[index]);
+	}
+
+	return vec;
+}
+
 /// <summary>
 /// Wrapper for creating a renderer of the specified type.
-/// First template argument expected to be float or double for CPU renderer,
-/// Second argument expected to be float or double for CPU renderer, and only float for OpenCL renderer.
 /// </summary>
 /// <param name="renderType">Type of renderer to create</param>
-/// <param name="platform">The index platform of the platform to use</param>
-/// <param name="device">The index device of the device to use</param>
+/// <param name="devices">The vector of platform/device indices to use</param>
 /// <param name="shared">True if shared with OpenGL, else false.</param>
 /// <param name="texId">The texture ID of the shared OpenGL texture if shared</param>
 /// <param name="errorReport">The error report for holding errors if anything goes wrong</param>
 /// <returns>A pointer to the created renderer if successful, else false.</returns>
-template <typename T, typename bucketT>
-static Renderer<T, bucketT>* CreateRenderer(eRendererType renderType, uint platform, uint device, bool shared, GLuint texId, EmberReport& errorReport)
+template <typename T>
+static Renderer<T, float>* CreateRenderer(eRendererType renderType, const vector<pair<size_t, size_t>>& devices, bool shared, GLuint texId, EmberReport& errorReport)
 {
 	string s;
-	unique_ptr<Renderer<T, bucketT>> renderer;
+	unique_ptr<Renderer<T, float>> renderer;
 
 	try
 	{
-		if (renderType == CPU_RENDERER)
-		{
-			s = "CPU";
-			renderer = unique_ptr<Renderer<T, bucketT>>(new Renderer<T, bucketT>());
-		}
-		else if (renderType == OPENCL_RENDERER)
+		if (renderType == OPENCL_RENDERER && !devices.empty())
 		{
 			s = "OpenCL";
-			renderer = unique_ptr<Renderer<T, bucketT>>(new RendererCL<T, bucketT>(platform, device, shared, texId));
+			renderer = unique_ptr<Renderer<T, float>>(new RendererCL<T, float>(devices, shared, texId));
 
 			if (!renderer.get() || !renderer->Ok())
 			{
@@ -267,9 +284,18 @@ static Renderer<T, bucketT>* CreateRenderer(eRendererType renderType, uint platf
 					errorReport.AddToReport(renderer->ErrorReport());
 
 				errorReport.AddToReport("Error initializing OpenCL renderer, using CPU renderer instead.");
-				renderer = unique_ptr<Renderer<T, bucketT>>(new Renderer<T, bucketT>());
+				renderer = unique_ptr<Renderer<T, float>>(new Renderer<T, float>());
 			}
 		}
+		else
+		{
+			s = "CPU";
+			renderer = unique_ptr<Renderer<T, float>>(new Renderer<T, float>());
+		}
+	}
+	catch (const std::exception& e)
+	{
+		errorReport.AddToReport("Error creating " + s + " renderer: " + e.what() + "\n");
 	}
 	catch (...)
 	{
@@ -279,6 +305,100 @@ static Renderer<T, bucketT>* CreateRenderer(eRendererType renderType, uint platf
 	return renderer.release();
 }
 
+/// <summary>
+/// Wrapper for creating a vector of renderers of the specified type for each passed in device.
+/// If shared is true, only the first renderer will be shared with OpenGL.
+/// Although a fallback GPU renderer will be created if a failure occurs, it doesn't really
+/// make sense since the concept of devices only applies to OpenCL renderers.
+/// </summary>
+/// <param name="renderType">Type of renderer to create</param>
+/// <param name="devices">The vector of platform/device indices to use</param>
+/// <param name="shared">True if shared with OpenGL, else false.</param>
+/// <param name="texId">The texture ID of the shared OpenGL texture if shared</param>
+/// <param name="errorReport">The error report for holding errors if anything goes wrong</param>
+/// <returns>The vector of created renderers if successful, else false.</returns>
+template <typename T>
+static vector<unique_ptr<Renderer<T, float>>> CreateRenderers(eRendererType renderType, const vector<pair<size_t, size_t>>& devices, bool shared, GLuint texId, EmberReport& errorReport)
+{
+	string s;
+	vector<unique_ptr<Renderer<T, float>>> v;
+
+	try
+	{
+		if (renderType == OPENCL_RENDERER && !devices.empty())
+		{
+			s = "OpenCL";
+			v.reserve(devices.size());
+
+			for (size_t i = 0; i < devices.size(); i++)
+			{
+				vector<pair<size_t, size_t>> tempDevices{ devices[i] };
+				auto renderer = unique_ptr<Renderer<T, float>>(new RendererCL<T, float>(tempDevices, !i ? shared : false, texId));
+
+				if (!renderer.get() || !renderer->Ok())
+				{
+					ostringstream os;
+
+					if (renderer.get())
+						errorReport.AddToReport(renderer->ErrorReport());
+
+					os << "Error initializing OpenCL renderer for platform " << devices[i].first << ", " << devices[i].second;
+					errorReport.AddToReport(os.str());
+				}
+				else
+					v.push_back(std::move(renderer));
+			}
+		}
+		else
+		{
+			s = "CPU";
+			v.push_back(std::move(unique_ptr<Renderer<T, float>>(::CreateRenderer<T>(CPU_RENDERER, devices, shared, texId, errorReport))));
+		}
+	}
+	catch (const std::exception& e)
+	{
+		errorReport.AddToReport("Error creating " + s + " renderer: " + e.what() + "\n");
+	}
+	catch (...)
+	{
+		errorReport.AddToReport("Error creating " + s + " renderer.\n");
+	}
+
+	if (v.empty() && s != "CPU")//OpenCL creation failed and CPU creation has not been attempted, so just create one CPU renderer and place it in the vector.
+	{
+		try
+		{
+			s = "CPU";
+			v.push_back(std::move(unique_ptr<Renderer<T, float>>(::CreateRenderer<T>(CPU_RENDERER, devices, shared, texId, errorReport))));
+		}
+		catch (const std::exception& e)
+		{
+			errorReport.AddToReport("Error creating fallback" + s + " renderer: " + e.what() + "\n");
+		}
+		catch (...)
+		{
+			errorReport.AddToReport("Error creating fallback " + s + " renderer.\n");
+		}
+	}
+
+	return v;
+}
+
+/// <summary>
+/// Perform a render which allows for using strips or not.
+/// If an error occurs while rendering any strip, the rendering process stops.
+/// </summary>
+/// <param name="renderer">The renderer to use</param>
+/// <param name="ember">The ember to render</param>
+/// <param name="finalImage">The vector to place the final output in</param>
+/// <param name="time">The time position to use, only valid for animation</param>
+/// <param name="strips">The number of strips to use. This must be validated before calling this function.</param>
+/// <param name="yAxisUp">True to flip the Y axis, else false.</param>
+/// <param name="perStripStart">Function called before the start of the rendering of each strip</param>
+/// <param name="perStripFinish">Function called after the end of the rendering of each strip</param>
+/// <param name="perStripError">Function called if there is an error rendering a strip</param>
+/// <param name="allStripsFinished">Function called when all strips successfully finish rendering</param>
+/// <returns>True if all rendering was successful, else false.</returns>
 template <typename T>
 static bool StripsRender(RendererBase* renderer, Ember<T>& ember, vector<byte>& finalImage, double time, size_t strips, bool yAxisUp,
 	std::function<void(size_t strip)> perStripStart,
@@ -354,6 +474,17 @@ static bool StripsRender(RendererBase* renderer, Ember<T>& ember, vector<byte>&
 	return success;
 }
 
+/// <summary>
+/// Verify that the specified number of strips is valid for the given height.
+/// The passed in error functions will be called if the number of strips needs
+/// to be modified for the given height.
+/// </summary>
+/// <param name="height">The height in pixels of the image to be rendered</param>
+/// <param name="strips">The number of strips to split the render into</param>
+/// <param name="stripError1">Function called if the number of strips exceeds the height of the image</param>
+/// <param name="stripError2">Function called if the number of strips does not divide evently into the height of the image</param>
+/// <param name="stripError3">Called if for any reason the number of strips used will differ from the value passed in</param>
+/// <returns>The actual number of strips that will be used</returns>
 static size_t VerifyStrips(size_t height, size_t strips,
 	std::function<void(const string& s)> stripError1,
 	std::function<void(const string& s)> stripError2,