diff --git a/docs/index.md b/docs/index.md index 00df82bbe..2697ac53a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -8,7 +8,7 @@ development and use. ## Libraries Yocto/GL is split into small libraries to make code navigation easier. -See each header file for documentation. +Here is a list of the main libraries. - [Yocto/Math](yocto/yocto_math.md): fixed-size vectors, matrices, rigid frames, transforms @@ -23,26 +23,28 @@ See each header file for documentation. functions, bsdf lobes, transmittance lobes, phase functions - [Yocto/Image](yocto/yocto_image.md): simple image data type, image resizing, tonemapping, color correction, procedural images, procedural sun-sky -- [Yocto/Shape](yocto/yocto_shape.md): utilities for manipulating - triangle meshes, quads meshes and line sets, computation of normals and - tangents, linear and Catmull-Clark subdivision, procedural shapes generation, - ray intersection and closest point queries +- [Yocto/Shape](yocto/yocto_shape.md): simple shape data structure, utilities + for manipulating triangle meshes, quads meshes and line sets, computation of + normals and tangents, linear and Catmull-Clark subdivision, + procedural shapes generation, ray intersection and closest point queries - [Yocto/Mesh](yocto/yocto_mesh.md): computational geometry utilities for triangle meshes, mesh geodesic, mesh cutting +- [Yocto/Scene](yocto/yocto_scene.md): scene representation and properties + evaluation - [Yocto/Bvh](yocto/yocto_bvh.md): ray intersection and closest point queries - of triangle meshes, quads meshes, line sets and instances scenes using a + of triangle meshes, quads meshes, line sets and shape instances using a two-level bounding volume hierarchy -- [Yocto/Scen](yocto/yocto_scene.md): scene representation and properties - evaluation -- [Yocto/SceneIO](yocto/yocto_sceneio.md): image serialization, - shape serialization, scene serialization, text and binary serialization, - path helpers - [Yocto/Trace](yocto/yocto_trace.md): path tracing of surfaces and hairs supporting area and environment illumination, microfacet GGX and subsurface scattering, multiple importance sampling -- [Yocto/ModelIO](yocto/yocto_modelio.md): parsing and writing for Ply, Obj, - Stl, Pbrt formats - [Yocto/Cli](yocto/yocto_cli.md): printing utilities and command line parsing +- [Yocto/ImageIO](yocto/yocto_imageio.md): image serialization +- [Yocto/ShapeIO](yocto/yocto_shapeio.md): shape serialization +- [Yocto/SceneIO](yocto/yocto_sceneio.md): scene serialization +- [Yocto/CommonIO](yocto/yocto_commonio.md): common IO operations, path helpers, + text and binary serialization, Json data model, Json serialization +- [Yocto/ModelIO](yocto/yocto_modelio.md): low-level parsing and writing for + Ply, Obj, Stl, Pbrt formats - [Yocto/Parallel](yocto/yocto_parallel.md): concurrency utilities ## Example Applications @@ -68,49 +70,41 @@ included in the [project site](https://xelatihy.github.io/yocto-gl/). Yocto/GL follows a "data-oriented programming model" that makes data explicit. Data is stored in simple structs and accessed with free functions or directly. All data is public, so we make no attempt at encapsulation. -Most objects is Yocto/GL have value semantic, while large data structures -use reference semantic with strict ownership. This means that everything -can be trivially serialized and there is no need for memory management. - We do this since this makes Yocto/GL easier to extend and quicker to learn, with a more explicit data flow that is easier when writing parallel code. Since Yocto/GL is mainly used for research and teaching, explicit data is both more hackable and easier to understand. +Nearly all objects in Yocto/GL have value semantic. This means that everything +can be trivially copied and serialized and there is no need for memory management. +While this has the drawback of potentially introducing spurious copies, it does +have the benefit of ensuring that no memory corruption can occur, which +turned out was a major issue for novice C++ users, even in a very small +library like this one. + In terms of code style we prefer a functional approach rather than an object oriented one, favoring free functions to class methods. All functions -and data are defined in sibling namespaces contained in the `yocto` namespace -so libraries can call all others, but have to do so explicitly. +and data are defined in the `yocto` namespace so libraries can call each others +trivially. The use of templates in Yocto was the reason for many refactoring, going from no template to heavy template use. At this point, Yocto uses some templates for readability. In the future, we will increase the use of templates in math code, while keeping many APIs explicitly typed. -We do not use exception for error reporting, but only to report "programmers" -errors. For example, IO operations use boolean flags and error strings for -human readable errors, while exceptions are used when preconditions or -post conditions are violated in functions. - -After several refactoring, we settled on a value-based approach, since the use -of pointers and reference semantics was hard for many of our users. While -this has the drawback of potentially introducing spurious copies, it does -have th benefit of ensuring that no memory corruption can occur, which -turned out was a major issue for novice C++ users, even in a very small -library like this one. - -## Credits - -Main contributors: +We support both exceptions and return codes for error reporting. For example, +we have two versions of many IO operations that either use boolean flags and +error strings for human readable errors, or throw exceptions with the same +error string. Internally exceptions are used, since they provide cleaner +code and where in fact faster ion low-level parsers. At the moment, +exceptions are only used in IO functions and to report "programmer" +errors, namely when preconditions or post conditions are violated in functions. -- Fabio Pellacini (lead developer): [web](http://pellacini.di.uniroma1.it), [github](https://github.com/xelatihy) -- Edoardo Carra: [github](https://github.com/edoardocarra) -- Giacomo Nazzaro: [github](https://github.com/giacomonazzaro) +## License -This library includes code from the [PCG random number generator](http://www.pcg-random.org), -boost `hash_combine`, and public domain code from `github.com/sgorsten/linalg`, -`gist.github.com/badboy/6267743`. -Other external libraries are included with their own license. +The library is released under the MIT license. We include various external +dependencies in the distribution that each have thir own license, compatible +with the chosen one. ## Compilation @@ -120,20 +114,18 @@ OsX (Xcode >= 11), Windows (MSVC 2019) and Linux (gcc >= 9, clang >= 9). You can build the example applications using CMake with `mkdir build; cd build; cmake ..; cmake --build` -Yocto/GL depends on `stb_image.h`, `stb_image_write.h`, `stb_image_resize.h` and -`tinyexr.h` for image loading, saving and resizing, `cgltf.h` and `json.hpp` -for glTF and JSON support, and `filesystem.hpp` to support C++17 filesystem API -when missing. All dependencies are included in the distribution. +Yocto/GL required dependencies are included in the distribution and do not +need to be installed separately. -Yocto/GL optionally supports building OpenGL demos, which are handled by including -glad, GLFW, ImGui as dependencies in apps. OpenGL support might eventually -become part of the Yocto/GL libraries. OpenGL support is enabled by defining -the cmake option `YOCTO_OPENGL` and contained in the `yocto_gui` library. +Yocto/GL optionally supports building OpenGL demos. OpenGL support is enabled +by defining the cmake option `YOCTO_OPENGL` and contained in the `yocto_gui` +library. OpenGL dependencies are included in this repo. Yocto/GL optionally supports the use of Intel's Embree for ray casting. See the main CMake file for how to link to it. Embree support is enabled by -defining the cmake option `YOCTO_EMBREE`. +defining the cmake option `YOCTO_EMBREE`. Embree needs to be installed separately. Yocto/GL optionally supports the use of Intel's Open Image Denoise for denoising. See the main CMake file for how to link to it. Open Image Denoise support -is enabled by defining the cmake option `YOCTO_DENOISE`. +is enabled by defining the cmake option `YOCTO_DENOISE`. +OIDN needs to be installed separately. diff --git a/docs/yocto/yocto_cli.md b/docs/yocto/yocto_cli.md index 1ac4a9f4f..89ea0f920 100644 --- a/docs/yocto/yocto_cli.md +++ b/docs/yocto/yocto_cli.md @@ -3,7 +3,7 @@ Yocto/Cli is a collection of utilities used in writing command-line applications, including parsing command line arguments, printing values, timers and progress bars. -Yocto/Cli is implemented in `yocto_cli.h`. +Yocto/Cli is implemented in `yocto_cli.h` and `yocto_cli.cpp`. ## Printing values @@ -42,13 +42,17 @@ In these commands, `name` is the option name, `value` is a reference to the variable that we want to set, `help` is a usage message, `alt` is the short option flag, and `req` determines whether the option is required, and `check` is either a range of valid values er a list of valid choices. -After adding all arguments, use `parse_cli(cli)` to parse the -command-line. If an error occurs, the parser exists. A help flag is also -handled in this function. +A help flag is automatically added. + +After adding all arguments, use `parse_cli(cli, args)` to parse the +command-line. If an error occurs, the parser throws a `cli_error` exception, +that includes a message that can be printed to inform the user. +If you cannot use exceptions, use the function `parse_cli(cli, args, error)` +that returns whether an error has occurred as a boolean and the error string. For positional arguments, the argument name is only used for printing help. -For optional arguments, the command line is parsed for options named `--` followed -by the option `name` or `-` followed by the `alt` short name. +For optional arguments, the command line is parsed for options named `--` +followed by the option `name` or `-` followed by the `alt` short name. The type of each option is determined by the passed reference. The parser supports integers, floating point numbers, strings, boolean flags, enums and arrays of strings. @@ -66,23 +70,78 @@ add_option(cli, "samples", samples, "samples"); // optional argument add_option(cli, "flag", flag, "flag"); // optional flag add_argument(cli, "scene", scene, "scene"); // positional argument add_option(cli, "scenes", scenes, "scenes"); // positional arguments -parse_cli(cli); +auto args = make_cli_args(argc, argv); // make a vector of args +try { + parse_cli(cli, args); // parse args +} catch(const cli_error& error) { + print_info(error.what()); // exit with error + exit(1); +} catch(const cli_help& help) { + print_info(help.what()); // exit with help + exit(0); +} catch (const std::exception& error) { + print_info(error.what()); // exit with other error + exit(1); +} ``` +You can avoid using writing the exception chain above by using the +`handle_errors(func)` or `handle_errors(func, args)` function that executes a +function and handle all errors internally. + +```cpp +void app(const vector& args) { ... } // parse cli and run +handle_errors(app, make_cli_args(argc, argv)); // run and handle errors +``` + +## Command-Line Commands + The command line parser also support commands, that are created by `add_command(cli, name, help)`, where `name` is the command name and `help` is the command usage message. You can add options to commands using the functions above. Commands can have any optional arguments, -but support either sub-commands or positional arguments. +but support either sub-commands or positional arguments. To get the selected +command, set a string variable with `set_command_var(cli, var)` that will +receive the command name. ```cpp -auto samples = 10; // state -auto scene = ""s; -auto cli = make_cli("app", "testing cli"); // initialize cli -auto& render = add_command(cli, "render", "render"); // command -add_option(render, "samples", samples, "samples"); // optional argument -add_argument(render, "scene", scene, "scene"); // positional argument -auto& convert = add_command(cli, "convert", "convert"); // command -add_argument(convert, "scene", scene, "scene"); // positional argument -parse_cli(cli); +auto samples = 10; auto scene = ""s; // state +auto command = string{}; // selected command +auto cli = make_cli("app", "testing cli"); // initialize cli +set_command_var(cli, command); // command variable +auto render = add_command(cli, "render", "render"); // command +add_option(render, "samples", samples, "samples"); // optional argument +add_argument(render, "scene", scene, "scene"); // positional argument +auto convert = add_command(cli, "convert", "convert"); // command +add_argument(convert, "scene", scene, "scene"); // positional argument +parse_cli(cli, make_cli_args(argc, argv)); // parse cli +if (command == "render") { ... } // execute commands +else if (command == "samples") { ... } ``` + +## Command-Line JSON Serialization + +Command-line arguments can be loaded from JSON config files by specifying the +`--config ` option in the command line. The semantic of the +config file is that configs are loaded first, and then overridden by +the value specified in the terminal. + +The JSON file should contain a value for each specified option, of the same +type as the option itself. For commands, use a JSON object to specify their +values. The command variable is called `command`. + +Alternatively, you can specify an option that, if specified, indicates a +filename in a directory where the config might be found, using +`add_option_with_config(cli, name, value, usage, config, alt)`. +Here `config` is the filename of the optional config file. + +## Command-Line Positional Arguments And Flags + +While for now positional arguments are supported, their use is deprecated. +The same is true for the use of short flags, like `-h`. +Th main reasons for this is to make the input to CLI tools more readable by +using long names for options and avoiding unnamed positional arguments. +This is also a better match for the JSON config files and since internally +the CLI uses JSON as a data model to process values. +In future releases, positional arguments will be deprecated and eventually +removed. \ No newline at end of file diff --git a/docs/yocto/yocto_commonio.md b/docs/yocto/yocto_commonio.md new file mode 100644 index 000000000..03cda94f3 --- /dev/null +++ b/docs/yocto/yocto_commonio.md @@ -0,0 +1,226 @@ +# Yocto/CommonIO: Common IO functionality + +Yocto/CommonIO is a collection of utilities used in writing IO functionality, +including file IO, Json IO, and path manipulation. +Yocto/CommonIO is implemented in `yocto_commonio.h` and `yocto_commonio.cpp`, +and depends on `json.hpp` for Json serialization and number printing, and +`fast_float.h` for number parsing. + +## Errors handling in IO functions + +IO functions in Yocto/GL have a dual interface to support their use with and +without exceptions. IO functions with exception are written as +`load_(filename, data, )` and +`save_(filename, data, )` where `` is the data type +read or written, and `` is an optional list of IO options. +A convenience shortcut is provided for all loading functions that returns the +data directly, written as `data = load_(filename, )`. +Upon errors, an `io_error` is thrown from all IO functions. +This makes the error-handling code more uniform across the library. +`io_error` has fields for `filename` and `message` that can retrieved directly, +and a message for users that can be retrieved with the `.what()` method. + +```cpp +auto text = string{}; +try { + load_text("input_file.txt", text); // load text + text = load_text("input_file.txt"); // alternative load + save_text("output_file.txt", text); // save text +} catch(const io_error& error) { + print_info(error.what()); exit(1); // handle error +} +``` + +IO functions without exceptions are written as +`load_(filename, data, error, )` and +`save_(filename, data, error, )` where `error` is a string +that contains a message if an error occurred. These functions return a boolean +flag that indicates whether the operation succeeded. + +```cpp +auto text = string{}; +auto error = string{}; +if(!load_text("input_file.txt", text)) // load text + print_info(error); exit(1); // handle error +if(!save_text("output_file.txt", text)) // save text + print_info(error); exit(1); // handle error +``` + +## Text and binary serialization + +Yocto/CommonIO supports reading and writing whole files of either binary +data or text. Use `load_text(filename, text)` or `text = load_text(filename)` +to load text, and `save_text(filename, text)` to save it. +Use `load_binary(filename, data)` or `data = load_binary(filename)` +to load text, and `save_binary(filename, data)` to save it. +Text is stored as a string and binary data is stored as an array of bytes. +Use without exception is supported as described above. + +```cpp +auto text = string{}; +load_text("input_file.txt", text); // load text +save_text("output_file.txt", text); // save text +auto text1 = load_text("input_file.txt"); // alternative load +auto data = vector{}; +load_binary("input_file.bin", data); // load data +save_binary("output_file.bin", data); // save data +auto data1 = load_binary("input_file.bin"); // alternative load +``` + +## Path manipulation utilities + +Yocto/CommonIO contains several helper function to manipulate paths. Paths +are encoded in UTF8 across the library and these functions make it easier to +handle UTF8-encoded paths across operating systems, by wrapping +`std::filesystem` with a string interface. + +Use `path_dirname(filename)`, `path_extension(filename)`, +`path_filename(filename)`, `path_basename(fillename)` +to extract the directory, extension, filename and basename from a path. +Use `path_join(patha,pathb)` to joins paths and +`replace_extension(filename,ext)` to replace a path extension. +Use `path_exists(filename)` to check if a path exists and +`path_isdir(filename)` and `path_isfile(filename)` to check whether +it is a directory ot file respectively. +Use `list_directory(dirname)` to list directory contents, +`make_directory(dirname)` to create a directory, and +`path_current()` to get the current directory. + +## Json data type + +Throughout the library, JSON data is used extensively as a simple and flexible +IO format. Most JSON libraries we reviewed use header-only formulation that +slow down compile time significantly. For this reason, we provide a simple +JSON data type modeled after `nlohmann::json`. Here we review quickly its +functionality, for those familiar with the JSON format. + +We model JSON values with the `json_value` type. Most functionality in this +type mimic the C++ `vector` and `map` functions. In fact, if you know how +to use those type and you know how to handle JSON data, then it should be +easy to use `json_value`. Json + +Json values are unions of either null, strings, booleans, numbers, arrays +of Json values and dictionaries of string to Json values. To avoid loosing +precision, we store numbers with 64 bits per value and as either signed +integers, unsigned integers, or floating point. Json arrays are stored as +`vector`, while Json dictionaries are stored as ordered maps, +which are roughly equivalent to `vector>`. +The choice off dictionary storage handle well small dictionaries, which is +mostly the case in Json. + +The type of a `json_value` is queried by the `json.type()` method that returns +a `json_type` value. More conveniently, you can query thee type with +`json.is_null()`, `json.is_integer()`, `json.is_number()`, `json.is_boolean()`, +`json.is_string()`,`json.is_array()`, and `json.is_object()`. +`json.is_integer()` checks if a number is signed or unsigned integer, +while `json.is_number()` returns true for any number type. + +You can convert values to a `json_value` by either constructing it, +assigning to it and calling the `json.set(value)` method with values of all +builtin types, arrays of builtin types and enums, if they define the trait +`json_enum_trait`. + +```cpp +auto json = json_value{5}; // construct a value of type integer +json = "text"; // converts to a value of type string +json.set(vector{...}); // converts to a value of type array +``` + +You can convert a `json_value` to values by either casting to the type of the +value or calling the `json.get(value)` and `value = json.get()` methods. +A `json_error` is thrown if the conversion cannot be done. + +```cpp +auto json = json_value{"hello"}; // construct a json_value +auto text = (string)json; // convert to string +json.get(text); // convert to string +``` + +Besides conversions, Json arrays and dictionaries can be manipulated directly. +Json arrays are constructed by assigning a `json_array` to a `json_value`, +either constructed directly or via the `json_value::array()` and +`json_value::array(size)`. Array sizes are queries via the `json.empty()` and +`json.size()` methods. Arrays can be resized with `json.resize(size)`, or by +appending values at the array end with `json.push_back(value)` and +`json.emplace_back(args...)`. Array items are accessed by index with the +`[index]` operator and the `json.at(index)` method. Arrays can be iterated +by using the `json.begin()` and `json.end()` methods. + +```cpp +auto json = json_value::array(); // construct a value containing an array +json.push_back(json_value{...}); // append a value +json.push_back(5); // append a value converted from an int +auto size = json.size(); // get size +auto converted = (int)json.at(5); // access value by index +for(auto& item : json) { ... } // iterate over the array +``` + +Json dictionaries are constructed by assigning a `json_object` to a `json_value`, +either constructed directly or via the `json_value::object()`. Object sizes +are queries via the `json.empty()` and `json.size()` methods. Values are +inserted in dictionaries by using the `[key]` operator, or using the +`insert_back(key, value)` method for a faster insertion that does not check +for duplicate keys. Object items are accessed by key with the `[key]` operator +and the `json.at(key)` method. The existence of a key is checked with the +`json.contains(key)` method. Json dictionaries are iterated with the +`json.items().begin()` and `json.items().end()`. +Missing keys are so common in Json that Json dictionaries support the +`json.try_get(key, value)` that assigns to the value only if the key is present, +and `json.get_or(key, default)` that returns the value if the key is present or +the default value specified. + +```cpp +auto json = json_value::object(); // construct a value containing a dict. +json["key"] = json_value{...}; // insert a value +json["key"] = 5; // insert a value converted from an int +auto size = json.size(); // get size +auto converted = (int)json.at("key"); // access value by key +for(auto& [key, item] : json.items) { ... } // iterate over the dictionary +auto value = json.get_or("key", 5); // get value or default +json.try_get("key", value); // get value if present +``` + +## Json serialization + +Use `load_json(filename, json)` or `json = load_json(filename)` to load json +data from a file, and `save_json(filename, json)` to save json data to a file. +Upon errors, an `io_error` is thrown from all IO functions. +See [Yocto/CommonIO](yocto_commonio.md) for discussion on error handling +and use without exceptions. + +```cpp +auto json = json_value{}; +load_json("input_file.json", json); // load json +save_json("output_file.json", json); // save json +auto json1 = load_json("input_file.txt"); // alternative load +``` + +Use `parse_json(text, json)` or `json = parse_json(text)` to parse json data +from a string, and `format_json(text, json)` or `text = format_json(json)` to +write json data to a string. +Use without exception is supported as described above. + +```cpp +auto input_string = string{...}; +auto json = json_value{}; +parse_json(input_string, json); // parse json +auto output_string = string{}; +format_json(output_string, json); // format json +auto json1 = parse_json(input_string); // alternative parse +auto output_string1 = format_json(json1); // alternative format +``` + +## Ordered map + +Json dictionaries are stored in an `ordered_map` structure. Ordered maps are +unsorted just arrays of key-value pairs, with methods added to match the +map data structures in the standard library, that we do not report here since +it would be a repetition. + +We use unsorted arrays for dictionaries since dictionary implementations +tradeoff memory consumption, number of allocation, and insertion and +query speeds. For th specific use of Json values in Yocto/GL, a good trade off +is to reduce memory pressure, while optimizing the lookups only for small +dictionaries, as is generally the case for the Json values in Yocto/GL. +One exception are large dictionaries that need to be only iterated. +In that case, we can skip the key checks and insert values at the end directly. diff --git a/docs/yocto/yocto_image.md b/docs/yocto/yocto_image.md index 6e7461490..3067c2f43 100644 --- a/docs/yocto/yocto_image.md +++ b/docs/yocto/yocto_image.md @@ -48,7 +48,9 @@ auto ch0 = eval_image(ldr, {0.5,0.5}); // samples in sRGB, returns as sRGB auto ch1 = eval_image(ldr, {0.5,0.5}, true); // treats sRGB values as linear ``` -Image loading and saving is defined in [Yocto/SceneIO](yocto_sceneio.md). +## Image serialization + +Image loading and saving is defined in [Yocto/ImageIO](yocto_imageio.md). ## Image utilities @@ -192,7 +194,7 @@ auto bordered = add_border(img, 1, {0, 0, 0, 1}); // add a thin black border Yocto/Image supports versions of the most of the above functions that work directly on pixel arrays, rather than the image structure. This low-level -interface may be helpful when building applications that mhave their own +interface may be helpful when building applications that have their own image data structure. In this interface we support arrays of `vec4f` pixels together with arrays of diff --git a/docs/yocto/yocto_imageio.md b/docs/yocto/yocto_imageio.md new file mode 100644 index 000000000..e9ab8d1f6 --- /dev/null +++ b/docs/yocto/yocto_imageio.md @@ -0,0 +1,23 @@ +# Yocto/ImageIO: Image serialization + +Yocto/ImageIO supports loading and saving images from Png, Jpg, Tga, Bmp, +Pnm, Hdr, Exr, Pfm. +Yocto/ImageIO is implemented in `yocto_imageio.h` and `yocto_imageio.cpp` +and depends on `stb_image.h`, `stb_image_write.h`, and `tinyexr.h`. + +## Image serialization + +Yocto/ImageIO supports reading and writing images. +Use `load_image(filename, image)` or `image = load_image(filename)` +to load images, and `save_image(filename, text)` to save it. +Images are stored as `image_data` structs defined in [Yocto/Image](yocto_image.md). +Upon errors, an `io_error` is thrown from all IO functions. +See [Yocto/CommonIO](yocto_commonio.md) for discussion on error handling +and use without exceptions. + +```cpp +auto image = image_data{}; +load_image("input_file.png", image); // load image +save_image("output_file.png", image); // save image +auto image1 = load_image("input_file.png"); // alternative load +``` diff --git a/docs/yocto/yocto_modelio.md b/docs/yocto/yocto_modelio.md index 4b298565c..9e90a5abe 100644 --- a/docs/yocto/yocto_modelio.md +++ b/docs/yocto/yocto_modelio.md @@ -4,6 +4,46 @@ Yocto/ModelIO is a collection of utilities for loading and saving scenes and meshes in Ply, Obj, Stl and Pbrt formats. Yocto/ModelIO is implemented in `yocto_modelio.h` and `yocto_modelio.cpp`. +## Errors handling in IO functions + +IO functions in Yocto/ModelIO have a dual interface to support their use with +and without exceptions. IO functions with exception are written as +`load_(filename, data, )` and +`save_(filename, data, )` where `` is the data type +read or written, and `` is an optional list of IO options. +A convenience shortcut is provided for all loading functions that returns the +data directly, written as `data = load_(filename, )`. +Upon errors, an `io_error` is thrown from all IO functions. +This makes the error-handling code more uniform across the library. +`io_error` has fields for `filename` and `message` that can retrieved directly, +and a message for users that can be retrieved with the `.what()` method. + +```cpp +auto ply = string{}; +try { + load_ply("input_file.ply", ply); // load ply + ply = load_ply("input_file.ply"); // alternative load + save_ply("output_file.ply", ply); // save ply +} catch(const io_error& error) { + print_info(error.what()); exit(1); // handle error +} +``` + +IO functions without exceptions are written as +`load_(filename, data, error, )` and +`save_(filename, data, error, )` where `error` is a string +that contains a message if an error occurred. These functions return a boolean +flag that indicates whether the operation succeeded. + +```cpp +auto ply = string{}; +auto error = string{}; +if(!load_ply("input_file.ply", ply)) // load ply + print_info(error); exit(1); // handle error +if(!save_ply("output_file.ply", ply)) // save ply + print_info(error); exit(1); // handle error +``` + ## Ply models The Ply file format is a generic file format used to serialize meshes and point @@ -20,20 +60,17 @@ All elements and properties are owned by the main `ply_model`. Yocto/ModelIO provides several functions to read and write Ply data whose use is preferred over direct data access. -Use `load_ply(filename, ply, error)` to load Ply files and -`save_ply(filename, ply, error)` to save them. -Both loading and saving take a filename, a reference to a Ply model, -and returns whether or not the file was loaded successfully. -In the case of an error, the IO functions set the `error` string with a -message suitable for displaying to a user. +Use `load_ply(filename, ply)` or `ply = load_ply(filename)` to load Ply files +and `save_ply(filename, ply)` to save them. +Both loading and saving take a filename, a reference to a model, and throw +an exception `io_error` exception if the file was not loaded successfully. +Use without exception is supported as described above. ```cpp -auto ply = ply_model{}; // ply model buffer -auto error = string{}; // error buffer -if(!load_ply(filename, ply, error)) // load ply - print_error(error); // check and print error -if(!save_ply(filename, ply, error)) // save ply - print_error(error); // check and print error +auto ply = ply_model{}; // ply model buffer +load_ply(filename, ply); // load ply +ply = load_ply(filename); // alternative load +save_ply(filename, ply; // save ply ``` ## Ply reading @@ -265,45 +302,38 @@ for(auto environment : obj.environments) // access environments [extension] print_info(environment.emission); // access environment properties ``` -Use `load_obj(filename, obj, error)` to load Obj files and -`save_obj(filename, obj, error)` to save them. -Both loading and saving take a filename, a reference to an Obj model, -and return whether or not the file was loaded successfully. -In the case of an error, the IO functions set the `error` string with a -message suitable for displaying to a user. +Use `load_obj(filename, obj)` or `obj = load_obj(filename)` to load Obj files +and `save_obj(filename, obj)` to save them. +Both loading and saving take a filename, a reference to a model, and throw +an exception `io_error` exception if the file was not loaded successfully. +Use without exception is supported as described above. Obj is a face-varying file format, while most applications handle only indexed meshes. The loading function takes an optional that specify whether to load as face-varying or convert to indexed meshes, which is the default. ```cpp -auto obj = obj_model{}; // obj model -auto error = string{}; // error -if(!load_obj(filename, obj, error)) // load obj as indexed meshes - print_error(error); // check and print error -if(!load_obj(filename, obj, error, true)) // load obj as face-varying - print_error(error); // check and print error -if(!save_obj(filename, obj, error)) // save obj - print_error(error); // check and print error +auto obj = obj_model{}; // obj model +load_obj(filename, obj); // load obj as indexed meshes +obj = load_obj(filename); // alternative load +load_obj(filename, obj, true); // load obj as face-varying +save_obj(filename, obj); // save obj ``` It is common in graphics to use Obj file to store single meshes. Yocto/Obj supports this modality by providing specialized loading and saving functions that take references to shapes as parameters. -Use `load_obj(filename, shape, error)` to load Obj shapes and -`save_obj(filename, shape, error)` to save them. The loading function takes +Use `load_obj(filename, shape)` or `shape = load_sobj(filename)` to load Obj +shapes and `save_obj(filename, shape)` to save them. The loading function takes an optional that specify whether to load as face-varying or convert to indexed meshes, which is the default. ```cpp -auto shape = obj_shape{}; // obj shape -auto error = string{}; // error -if(!load_obj(filename, shape, error)) // load obj as indexed meshes - print_error(error); // check and print error -if(!load_obj(filename, shape, error, true)) // load obj as face-varying - print_error(error); // check and print error -if(!save_obj(filename, shape, error)) // save obj - print_error(error); // check and print error +auto shape = obj_shape{}; // obj shape +load_obj(filename, shape) // load obj as indexed meshes +shape = load_sobj(filename); // alternative load +load_obj(filename, shape, true); // load obj as face-varying +save_obj(filename, shape); // save obj ``` ## Obj reading @@ -336,19 +366,18 @@ materials tags. ```cpp auto obj = obj_model{}; // obj model buffer -auto error = string{}; // error buffer -load_obj(filename, obj, error); // load obj +load_obj(filename, obj); // load obj auto& shape = obj.shapes.front(); // get shape -auto positions = vector{}; // vertex properties +auto positions = vector{}; // vertex properties get_positions(shape, positions); auto normals = vector{}; get_normals(shape, normals); auto texcoords = vector{}; get_texcoords(shape, texcoords); -auto triangles = vector{}; // element data +auto triangles = vector{}; // element data auto quads = vector{}; -auto materials = vector{}; // per-face material ids +auto materials = vector{}; // per-face material ids if(has_quads(shape)) { get_triangles(shape, triangles, materials); // read as triangles } else { @@ -369,8 +398,7 @@ face-varying. ```cpp auto obj = new obj_model{}; // obj model buffer -auto error = string{}; // error buffer -load_obj(filename, obj, error, true); // load obj as face-varying +load_obj(filename, obj, true); // load obj as face-varying auto& shape = obj.shapes.front(); // get shape auto positions = vector{}; // vertex properties @@ -459,8 +487,58 @@ add_positions(shape, positions); add_normals(shape, normals); add_texcoords(shape, texcoords); -auto error = string{}; // error buffer -save_obj(filename, obj, error); // save obj +save_obj(filename, obj); // save obj +``` + +## Stl models + +The Stl file format is a file format used to serialize meshes. +To use this library is helpful to understand the basic of the Stl +file format for example from the +[Stl Wikipedia page](). + +Yocto/ModelIO represents Stl data with the `stl_model` struct. +Stl models are defined as collections of shapes, stored in `stl_shape`. +Shapes are defined by an array of vertex positions, an array of triangle +indices, and an array of triangle normals. + +Use `load_stl(filename, stl)` or `stl = load_stl(filename)` to load Stl files +and `save_stl(filename, stl)` to save them. +Both loading and saving take a filename, a reference to a model, and throw +an exception `io_error` exception if the file was not loaded successfully. +Use without exception is supported as described above. + +```cpp +auto stl = stl_model{}; // stl model +load_stl(filename, stl); // load stl +stl = load_stl(filename); // alternative load +save_stl(filename, stl); // save stl +``` + +## Stl reading + +You can access Stl data directly from the shapes, without any further complexity. + +```cpp +auto stl = stl_model{}; // stl model buffer +load_obj(filename, stl); // load stl +auto& shape = stl.shapes.front(); // get shape + +auto positions = shape.positions; // vertex properties +auto triangles = shape.triangles; // element data +``` + +## Stl writing + +Similarly, you can create an Stl model by acting directly on the data. + +```cpp +auto stl = stl_model{}; // stl model buffer +auto shape = stl_shape{}; // create shape +shape.positions = vector{...}; +shape.triangles = vector{...}; +stl.shapes.push_back(shape); // add shape +load_obj(filename, stl); // save stl ``` ## Pbrt models @@ -511,18 +589,15 @@ for(auto environment : pbrt.environments) // access environments [extension] print_info(environment.emission); // access environment properties ``` -Use `load_pbrt(filename, pbrt, error)` to load Pbrt files and -`save_pbrt(filename, pbrt, error)` to save them. -Both loading and saving take a filename, a pointer to a Pbrt model, -and returns whether or not the file was loaded successfully. -In the case of an error, the IO functions set the `error` string with a -message suitable for displaying to a user. +Use `load_pbrt(filename, pbrt)` or `pbrt = load_pbrt(filename)` to load Pbrt +files and `save_pbrt(filename, pbrt)` to save them. +Both loading and saving take a filename, a reference to a model, and throw +an exception `io_error` exception if the file was not loaded successfully. +Use without exception is supported as described above. ```cpp -auto pbrt = new pbrt_scene{}; // obj model buffer -auto error = string{}; // error buffer -if(!load_pbrt(filename, pbrt, error)) // load obj - print_error(error); // check and print error -if(!save_pbrt(filename, pbrt, error)) // save obj - print_error(error); // check and print error +auto pbrt = pbrt_scene{}; // pbrt model buffer +load_pbrt(filename, pbrt); // load pbrt +pbrt = load_pbrt(filename); // alternative load +save_pbrt(filename, pbrt); // save pbrt ``` diff --git a/docs/yocto/yocto_scene.md b/docs/yocto/yocto_scene.md index 559b5cb56..b375811eb 100644 --- a/docs/yocto/yocto_scene.md +++ b/docs/yocto/yocto_scene.md @@ -174,7 +174,8 @@ auto envi = eval_environment(environment, dir); // eval environment ## Shapes Shapes, represented by `shape_data`, are indexed meshes of elements. -Shapes can contain only one type of element, either +Shapes are defined in [Yocto/Shape](yocto_shape.md), and briefly described +here for convenience. Shapes can contain only one type of element, either points, lines, triangles or quads. Shape elements are parametrized as in [Yocto/Geometry](yocto_geometry.md). Vertex properties are defined as separate arrays and include @@ -240,7 +241,7 @@ following materials: - `matte`, for materials like concrete or stucco, implemented as a lambertian bsdf; - `glossy`, for materials like plastic or painted wood, implemented as the sum of a lambertian and a microfacet dielectric lobe; -- `metallic`, for materials like metals, implemented as either a delta or +- `reflective`, for materials like metals, implemented as either a delta or microfacet brdf lobe; - `transparent`, for materials for thin glass, implemented as a delta or microfacet transmission bsdf; @@ -248,7 +249,7 @@ following materials: microfacet refraction bsdf; also support homogenous volume scattering; - `subsurface`, for materials for skin, implemented as a microfacet refraction bsdf with homogenous volume scattering - for no this is like `refractive`; -- `volume`, for materials like homogeneous smoke or fog, implemented as the lack +- `volumetric`, for materials like homogeneous smoke or fog, implemented as the lack of a surface interface but with volumetric scattering. - `gltfpbr`, for materials that range from glossy to metallic, implemented as the sum of a lambertian and a microfacet dielectric lobe; @@ -263,7 +264,7 @@ the index of refraction `ior`. The physical meaning of each parameter depends on the material type. By default surfaces are fully opaque, but can defined a `opacity` parameter and texture to define the surface coverage. -Materials like `refractive`, `subsurface` and `volume` may also specify +Materials like `refractive`, `subsurface` and `volumetric` may also specify volumetric properties. In these cases, the `color` parameter controls the volume density, while the `scattering` also define volumetric scattering properties by setting a `transmission` parameter controls the homogenous volume scattering. @@ -284,10 +285,10 @@ glossy.type = material_type::glossy; glossy.color = {0.5,1,0.5}; // with constant color glossyv.roughness = 0.1; // base roughness and a glossy.roughness_tex = texture_id; // roughness texture -auto metallic = material_data{}; // create a metallic material -glossy.type = material_type::metallic -metal.color = {0.5,0.5,1}; // constant color -metal.roughness = 0.1; // constant roughness +auto reflective = material_data{}; // create a reflective material +glossy.type = material_type::reflective +reflective.color = {0.5,0.5,1}; // constant color +reflective.roughness = 0.1; // constant roughness auto tglass = material_data{}; // create a transparent material tglass.type = material_type::transparent; tglass.color = {1,1,1}; // with constant color @@ -386,20 +387,6 @@ or `tesselate_subdivs(scene)` for the whole scene. tesselate_subdivs(scene); // tesselate all subdivs in the scene ``` -## Face-Varying shapes - -We also support standalone face-varying shapes, that are not stored in the scene -(see subdivs above). In this case, set the quads for positions, -normals and texture coordinates. - -```cpp -auto shape = fvshape_data{}; // create a shape -shape.quadspos = vector{...}; // set face-varying indices -shape.quadstexcoord = vector{...}; // for positions and textures -shape.positions = vector{...}; // set positions -shape.texcoords = vector{...}; // set texture coordinates -``` - ## Example scenes Yocto/Scene has a function to create a simple Cornell Box scene for testing. @@ -408,128 +395,3 @@ There are plans to increase support for more test scenes in the future. ```cpp auto scene = make_cornellbox(); // make cornell box ``` - -## Procedural shapes - -Yocto/Scene has convenience function to create various procedural shapes, -both for testing and for use in shape creation. These are wrappers to the -corresponding functions in [Yocto/Shape](yocto_shape.md), where we maintain -a comprehensive list of all procedural shapes supported. - -Procedural shapes take as input the desired shape resolution, the shape scale, -the uv scale, and additional parameters specific to that procedural shape. -These functions return a quad mesh, stored as a `shape_data` struct. -Use `make_rect(...)` for a rectangle in the XY plane, -`make_bulged_rect(...)` for a bulged rectangle, -`make_recty(...)` for a rectangle in the XZ plane, -`make_bulged_recty(...)` for a bulged rectangle in the XZ plane, -`make_box(...)` for a box, -`make_rounded_box(...)` for a rounded box, -`make_floor(...)` for a floor in the XZ plane, -`make_bent_floor(...)` for a bent floor, -`make_sphere(...)` for a sphere obtained from a cube, -`make_uvsphere(...)` for a sphere tessellated along its uvs, -`make_capped_uvsphere(...)` for a sphere with flipped caps, -`make_disk(...)` for a disk obtained from a quad, -`make_bulged_disk(...)` for a bulged disk, -`make_uvdisk(...)` for a disk tessellated along its uvs, -`make_uvcylinder(...)` for a cylinder tessellated along its uvs, -`make_rounded_uvcylinder(...)` for a rounded cylinder. - -```cpp -// make shapes with 32 steps in resolution and scale of 1 -auto shape_01 = make_rect({32,32}, {1,1}); -auto shape_02 = make_bulged_rect({32,32}, {1,1}); -auto shape_03 = make_recty({32,32}, {1,1}); -auto shape_04 = make_box({32,32,32}, {1,1,1}); -auto shape_05 = make_rounded_box({32,32,32}, {1,1,1}); -auto shape_06 = make_floor({32,32}, {10,10}); -auto shape_07 = make_bent_floor({32,32}, {10,10}); -auto shape_08 = make_sphere(32, 1); -auto shape_09 = make_uvsphere({32,32}, 1); -auto shape_10 = make_capped_uvsphere({32,32}, 1); -auto shape_11 = make_disk(32, 1); -auto shape_12 = make_bulged_disk(32, 1); -auto shape_13 = make_uvdiskm({32,32}, 1); -auto shape_14 = make_uvcylinder({32,32,32}, {1,1}); -auto shape_15 = make_rounded_uvcylinder({32,32,32}, {1,1}); -``` - -Yocto/Shape defines a few procedural face-varying shapes with similar interfaces -to the above functions. In this case, the functions return face-varying quads -packed in a `fvshape_data` struct. -Use `make_fvrect(...)` for a rectangle in the XY plane, -`make_fvbox(...)` for a box, -`make_fvsphere(...)` for a sphere obtained from a cube. - -```cpp -// make face-varying shapes with 32 steps in resolution and scale of 1 -auto fvshape_01 = make_fvrect({32,32}, {1,1}); -auto fvshape_02 = make_fvbox({32,32,32}, {1,1,1}); -auto fvshape_03 = make_fvsphere(32, 1); -``` - -Yocto/Shape provides functions to create predefined shapes helpful in testing. -These functions take only a scale and often provide only the positions as -vertex data. These functions return either triangles, quads, or -face-varying quads in a `shape_data` or `fvshape_data` struct. -Use `make_monkey(...)` for the Blender monkey as quads and positions only, -`make_quad(...)` for a simple quad, -`make_quady(...)` for a simple quad in the XZ plane, -`make_cube(...)` for a simple cube as quads and positions only, -`make_fvcube(...)` for a simple face-varying unit cube, -`make_geosphere(...)` for a geodesic sphere as triangles and positions only. -These functions return a `shape_data` or `fvshape_data`. - -```cpp -auto monkey = make_monkey(1); -auto quad = make_quad(1); -auto quady = make_quady(1); -auto cube = make_cube(1); -auto geosph = make_geosphere(1); -auto fvcube = make_fvcube(1); -``` - -Yocto/Shape supports the generation of points and lines sets. -Use `make_lines(...)` to create a line set in the XY plane, -`make_points(...)` for a collection of points at the origin, -adn `make_random_points(...)` for a point set randomly placed in a box. -These functions return points or lines, packed in a `shape_data` struct. - -```cpp -auto lines_01 = make_lines({4, 65536}, // line steps and number of lines - {1, 1}, {1, 1}, // line set scale and uvscale - {0.001, 0.001}); // radius at the bottom and top -// procedural points return points, positions, normals, texcoords, radia -auto [points, positions, normals, texcoords, radius] = make_points(65536); -auto points_01 = make_points(65536, // number of points - 1, // uvscale - 0.001); // point radius -auto points_02 = make_random_points(65536, // number of points - {1, 1, 1}, 1, // line set scale and uvscale - 0.001); // point radius -``` - -Yocto/Shape also defines a simple functions to generate randomized hairs -on a triangle or quad mesh. Use `make_hair(...)` to create a hair shape -from a triangle and quad mesh, and return a line set. - -```cpp -// Make a hair ball around a shape -auto lines = make_hair( - make_sphere(), // sampled surface - {8, 65536}, // steps: line steps and number of lines - {0.1, 0.1}, // length: minimum and maximum length - {0.001, 0.001}, // radius: minimum and maximum radius from base to tip - {0, 10}, // noise: noise added to hair (strength/scale) - {0, 128}, // clump: clump added to hair (strength/number) - {0, 0}); // rotation: rotation added to hair (angle/strength) -``` - -Finally, Yocto/Shape defines a function to create a quad mesh from a heighfield. -Use `make_heightfield(...)` to create a heightfield meshes. - -```cpp -auto heightfield = vctor{...}; // heightfield data -auto shape = make_heightfield(size, heightfield); // make heightfield mesh -``` diff --git a/docs/yocto/yocto_sceneio.md b/docs/yocto/yocto_sceneio.md index a0ccc16a7..630f024d8 100644 --- a/docs/yocto/yocto_sceneio.md +++ b/docs/yocto/yocto_sceneio.md @@ -114,43 +114,3 @@ if(!load_texture(filename, texture, error)) // load texture if(!save_texture(filename, texture, error)) // save texture print_error(error); // check and print error ``` - -## Text and binary serialization - -Text files are loaded with `load_text(filename, text, error)` and saved with -`save_text(filename, text, error)`. -Binary files are loaded with `load_binary(filename, binary, error)` and saved -with `save_binary(filename, binary, error)`. -Both loading and saving take a filename, a text or binary buffer, and return -whether or not the file was loaded successfully. -In the case of an error, the IO functions set the `error` string with a -message suitable for displaying to a user. - -```cpp -auto error = string{}; // error buffer -auto text = string{}; // text buffer -if(!load_text(filename, text, error)) // load a text file - print_error(error); // check and print error -if(!save_text(filename, text, error)) // save a text file - print_error(error); // check and print error -auto data = vector{}; // data buffer -if(!load_binary(filename, data, error)) // load a binary file - print_error(error); // check and print error -if(!save_binary(filename, data, error)) // save a binary file - print_error(error); // check and print error -``` - -## Path utilities - -Yocto/SceneIO contains several helper function to manipulate paths. These are -just convenience wrapper of `std::filesystem`. -Use `path_dirname(filename)`, `path_extension(filename)`, -`path_filename(filename)`, `path_basename(fillename)` -to extract the directory, extension, filename and basename from a path. -Use `path_join(patha,pathb)` to joins paths and -`replace_extension(filename,ext)` to replace a path extension. -Use `path_exists(filename)` to check if a path exists and -`path_isdir(filename)` and `path_isfile(filename)` to check whether -it is a directory ot file respectively. -Use `list_directory(dirname)` to list directory contents, and -`path_current()` to get the current directory. diff --git a/docs/yocto/yocto_shading.md b/docs/yocto/yocto_shading.md index c8cd6be88..d58fed86a 100644 --- a/docs/yocto/yocto_shading.md +++ b/docs/yocto/yocto_shading.md @@ -125,7 +125,7 @@ Yocto/Shading supports the following materials: - `matte`: matte appearance implemented as a diffuse bsdf - `glossy`: glossy appearance implemented as a sum of diffuse and microfacet bsdfs -- `metallic`: metallic appearance implemented as a delta or microfacet bsdfs +- `reflective`: metallic appearance implemented as a delta or microfacet bsdfs - `transparent`: thin glass-like appearance brdf implemented as a delta or microfacet bsdf - `refractive`: glass-like appearance implemented as a delta or microfacet bsdf - `passthrough`: used in volume rendering to simulated the absence of an interface @@ -141,15 +141,15 @@ auto ior = float{1.5}; // dielectric ior // evaluate smooth lobes auto b1 = eval_matte(color, normal, outgoing, incoming); auto b3 = eval_glossy(color, ior, roughness, normal, outgoing, incoming); -auto b4 = eval_metallic(color, roughness, normal, outgoing, incoming); +auto b4 = eval_reflective(color, roughness, normal, outgoing, incoming); auto b5 = eval_transparent(color, ior, roughness, normal, outgoing, incoming); auto b6 = eval_refractive(color, ior, roughness, normal, outgoing, incoming); // sample smooth lobes auto incoming1 = sample_matte(color, normal, outgoing, rand2f(rng)); auto pdf1 = sample_matte_pdf(color, normal, outgoing, incoming) // eval and sample delta lobes -auto incoming2 = sample_metallic(color, normal, outgoing); -auto b7 = eval_metallic(color, normal, outgoing, incoming); +auto incoming2 = sample_reflective(color, normal, outgoing); +auto b7 = eval_reflective(color, normal, outgoing, incoming); ``` ## Design considerations diff --git a/docs/yocto/yocto_shape.md b/docs/yocto/yocto_shape.md index bc1a14e48..5d7e71e90 100644 --- a/docs/yocto/yocto_shape.md +++ b/docs/yocto/yocto_shape.md @@ -7,18 +7,224 @@ Yocto/Shape is implemented in `yocto_shape.h` and `yocto_shape.cpp`. ## Shape representation Yocto/Shape supports shapes defined as collection of either points, lines, -triangles and quads. Most functions have overrides for all element types -when appropriate. - -Shapes are represented as indexed meshes, with arbitrary properties for -each vertex. Each vertex property is stored as a separate array, +triangles and quads. Shapes are represented as indexed meshes, with arbitrary +properties for each vertex. Each vertex property is stored as a separate array, and shape elements are stored as arrays of indices to faces. For element parametrization, we follow [Yocto/Geometry](yocto_geometry.md). +Shapes are represented as a simple struct called `shape_data`, that stores +the shape elements and the vertex properties as individual arrays. +Shapes can contain only one type of element, either points, lines, triangles or quads. +Vertex properties are defined as separate arrays and include +positions, normals, texture coords, colors, radius and tangent spaces. Vertex data is stored as `vector`, while element indices are stored as `vector`, `vector`, `vector`, `vector` for triangle meshes, quad meshes, line sets and point sets respectively. +For shapes, you should set the shape elements, i.e. point, limes, triangles +or quads, and the vertex properties, i.e. positions, normals, texture +coordinates, colors and radia. Shapes support only one element type. + +```cpp +auto shape = shape_data{}; // create a shape +shape.triangles = vector{...}; // set triangle indices +shape.positions = vector{...}; // set positions +shape.normals = vector{...}; // set normals +shape.texcoords = vector{...}; // set texture coordinates +``` + +Additionally, Yocto/Scene supports face-varying shape, as `fvshape_data`, +where each vertex data has its own topology. For now, only face-varying quad +meshes are supported since fave-varying shapes are used mostly with +Catmull-Clark subdivision. + +```cpp +auto shape = fvshape_data{}; // create a shape +shape.quadspos = vector{...}; // set face-varying indices +shape.quadstexcoord = vector{...}; // for positions and textures +shape.positions = vector{...}; // set positions +shape.texcoords = vector{...}; // set texture coordinates +``` + +Conversion to and from face-varying shapes is handled by the +`shape_to_fvshape(shape)` and `fvshape_to_shape(fvshape)` functions. +Note that conversion to an indexed shape is lossy since topology information is +lost and vertices may be duplicated. + +## Shape serialization + +Shape loading and saving is defined in [Yocto/ShapeIO](yocto_shapeio.md). + +## Shape properties + +Several functions are defined to evaluate the geometric properties of points +of shapes, indicated by the shape element id and, when needed, the shape element +barycentric coordinates. +Use `eval_position(...)` to evaluate the point position, +`eval_normal(...)` to evaluate the interpolate point normal, +`eval_texcoord(...)` to evaluate the point texture coordinates, +`eval_element_normal(...)` to evaluate the point geometric normal, and +`eval_color(...)` to evaluate the interpolate point color. + +```cpp +auto eid = 0; auto euv = vec3f{0.5,0.5}; // element id and uvs +auto pos = eval_position(shape, eid, euv); // eval point position +auto norm = eval_normal(shape, eid, euv); // eval point normal +auto st = eval_texcoord(shape, eid, euv); // eval point texture coords +auto col = eval_color(shape, eid, euv); // eval point color +auto gn = eval_element_normal(shape, eid, euv); // eval geometric normal +``` + +For shapes, we also support the computation of smooth vertex normals with +`compute_normals(shape)` and converting to and from face-varying representations +with `shape_to_fvshape(shape)` and `fvshape_to_shape(fvshape)`. + +## Shape sampling + +Shape support random sampling with a uniform distribution using +`sample_shape(...)` and `sample_shape_cdf(shape)`. Sampling works for lines and +triangles in all cases, while for quad it requires that the elements +are rectangular. + +```cpp +auto cdf = sample_shape_cdfd(shape); // compute the shape CDF +auto points = sample_shape(shape, cdf, num); // sample many points +auto point = sample_shape(shape, cdf, // sample a single point + rand1f(rng), rand2f(rng)); +``` + +## Procedural shapes + +Yocto/Scene has convenience function to create various procedural shapes, +both for testing and for use in shape creation. These are wrappers to the +corresponding functions in [Yocto/Shape](yocto_shape.md), where we maintain +a comprehensive list of all procedural shapes supported. + +Procedural shapes take as input the desired shape resolution, the shape scale, +the uv scale, and additional parameters specific to that procedural shape. +These functions return a quad mesh, stored as a `shape_data` struct. +Use `make_rect(...)` for a rectangle in the XY plane, +`make_bulged_rect(...)` for a bulged rectangle, +`make_recty(...)` for a rectangle in the XZ plane, +`make_bulged_recty(...)` for a bulged rectangle in the XZ plane, +`make_box(...)` for a box, +`make_rounded_box(...)` for a rounded box, +`make_floor(...)` for a floor in the XZ plane, +`make_bent_floor(...)` for a bent floor, +`make_sphere(...)` for a sphere obtained from a cube, +`make_uvsphere(...)` for a sphere tessellated along its uvs, +`make_capped_uvsphere(...)` for a sphere with flipped caps, +`make_disk(...)` for a disk obtained from a quad, +`make_bulged_disk(...)` for a bulged disk, +`make_uvdisk(...)` for a disk tessellated along its uvs, +`make_uvcylinder(...)` for a cylinder tessellated along its uvs, +`make_rounded_uvcylinder(...)` for a rounded cylinder. + +```cpp +// make shapes with 32 steps in resolution and scale of 1 +auto shape_01 = make_rect({32,32}, {1,1}); +auto shape_02 = make_bulged_rect({32,32}, {1,1}); +auto shape_03 = make_recty({32,32}, {1,1}); +auto shape_04 = make_box({32,32,32}, {1,1,1}); +auto shape_05 = make_rounded_box({32,32,32}, {1,1,1}); +auto shape_06 = make_floor({32,32}, {10,10}); +auto shape_07 = make_bent_floor({32,32}, {10,10}); +auto shape_08 = make_sphere(32, 1); +auto shape_09 = make_uvsphere({32,32}, 1); +auto shape_10 = make_capped_uvsphere({32,32}, 1); +auto shape_11 = make_disk(32, 1); +auto shape_12 = make_bulged_disk(32, 1); +auto shape_13 = make_uvdiskm({32,32}, 1); +auto shape_14 = make_uvcylinder({32,32,32}, {1,1}); +auto shape_15 = make_rounded_uvcylinder({32,32,32}, {1,1}); +``` + +Yocto/Shape defines a few procedural face-varying shapes with similar interfaces +to the above functions. In this case, the functions return face-varying quads +packed in a `fvshape_data` struct. +Use `make_fvrect(...)` for a rectangle in the XY plane, +`make_fvbox(...)` for a box, +`make_fvsphere(...)` for a sphere obtained from a cube. + +```cpp +// make face-varying shapes with 32 steps in resolution and scale of 1 +auto fvshape_01 = make_fvrect({32,32}, {1,1}); +auto fvshape_02 = make_fvbox({32,32,32}, {1,1,1}); +auto fvshape_03 = make_fvsphere(32, 1); +``` + +Yocto/Shape provides functions to create predefined shapes helpful in testing. +These functions take only a scale and often provide only the positions as +vertex data. These functions return either triangles, quads, or +face-varying quads in a `shape_data` or `fvshape_data` struct. +Use `make_monkey(...)` for the Blender monkey as quads and positions only, +`make_quad(...)` for a simple quad, +`make_quady(...)` for a simple quad in the XZ plane, +`make_cube(...)` for a simple cube as quads and positions only, +`make_fvcube(...)` for a simple face-varying unit cube, +`make_geosphere(...)` for a geodesic sphere as triangles and positions only. +These functions return a `shape_data` or `fvshape_data`. + +```cpp +auto monkey = make_monkey(1); +auto quad = make_quad(1); +auto quady = make_quady(1); +auto cube = make_cube(1); +auto geosph = make_geosphere(1); +auto fvcube = make_fvcube(1); +``` + +Yocto/Shape supports the generation of points and lines sets. +Use `make_lines(...)` to create a line set in the XY plane, +`make_points(...)` for a collection of points at the origin, +adn `make_random_points(...)` for a point set randomly placed in a box. +These functions return points or lines, packed in a `shape_data` struct. + +```cpp +auto lines_01 = make_lines({4, 65536}, // line steps and number of lines + {1, 1}, {1, 1}, // line set scale and uvscale + {0.001, 0.001}); // radius at the bottom and top +// procedural points return points, positions, normals, texcoords, radia +auto [points, positions, normals, texcoords, radius] = make_points(65536); +auto points_01 = make_points(65536, // number of points + 1, // uvscale + 0.001); // point radius +auto points_02 = make_random_points(65536, // number of points + {1, 1, 1}, 1, // line set scale and uvscale + 0.001); // point radius +``` + +Yocto/Shape also defines a simple functions to generate randomized hairs +on a triangle or quad mesh. Use `make_hair(...)` to create a hair shape +from a triangle and quad mesh, and return a line set. + +```cpp +// Make a hair ball around a shape +auto lines = make_hair( + make_sphere(), // sampled surface + {8, 65536}, // steps: line steps and number of lines + {0.1, 0.1}, // length: minimum and maximum length + {0.001, 0.001}, // radius: minimum and maximum radius from base to tip + {0, 10}, // noise: noise added to hair (strength/scale) + {0, 128}, // clump: clump added to hair (strength/number) + {0, 0}); // rotation: rotation added to hair (angle/strength) +``` + +Finally, Yocto/Shape defines a function to create a quad mesh from a heighfield. +Use `make_heightfield(...)` to create a heightfield meshes. + +```cpp +auto heightfield = vctor{...}; // heightfield data +auto shape = make_heightfield(size, heightfield); // make heightfield mesh +``` + +## Low-level interface: Shape representation + +Yocto/Shape also support an interface where single arrays are passed as opposed +to shape structs. This functionality will slowly be phased out and moved to +the higher level interface in future releases. Here we give example of the +low-level interface. + ```cpp auto triangles = vector{...}; // triangle indices auto positions = vector{...}; // vertex positions @@ -37,20 +243,7 @@ auto quadstexcoords = vector{...}; // quads indices for uvs auto texcoords = vector{...}; // vertex uvs ``` -Throughout the library, functions may either take index and vertex arrays -directly as input and output, or may pack these array in structs if deemed -appropriate. This design tries to balance readability and generality, without -forcing a single convention that would not be appropriate everywhere. - -If a higher level design is needed, [Yocto/Scene](yocto_scene.md) contains -the standalone types `shape_data` and `fvshape_data` to store indexed -and face-varying shapes respectively, a a collection of methods to work -on these type, that are essentially wrappers to the functionality in this -library. - -Shape loading and saving is defined in [Yocto/SceneIO](yocto_sceneio.md). - -## Vertex properties +## Low-level interface: Vertex properties Yocto/Shape provides many facilities to compute vertex properties for indexed elements. Use `triangles_normals(...)` and `quads_normals(...)` to compute @@ -74,7 +267,7 @@ auto [skinned_pos, skinned_norm] = skin_vertices(positions, normals, weights, joints, frames); // skinned positions ans normals ``` -## Flipping and aligning +## Low-level interface: Flipping and aligning Yocto/Shape provides functions to correct shapes that have inconsistent orientations or normals. Use `flip_normals(normals)` to flip all mesh normals. @@ -94,7 +287,7 @@ normals = flip_normals(normals); // flip normals positions = align_vertices(positions, {0,1,0}); ``` -## Edges and adjacencies +## Low-level interface: Edges and adjacencies Use `get_edges(triangles)` amd `get_edges(quads)` to get a list of unique edges for a triangle or quads mesh. @@ -120,7 +313,7 @@ for(auto& edge : edges) // iterate over edges auto boundary = get_boundary(emap); // get unsorted boundary edges ``` -## Ray-intersection and point-overlap +## Low-level interface: Ray-intersection and point-overlap Yocto/Shape provides ray-scene intersection for points, lines, triangles and quads accelerated by a BVH data structure. Our BVH is written for @@ -197,7 +390,7 @@ positions[...] = {...}; // update positions update_triangles_bvh(bvh, triangles, positions); // update BVH ``` -## Nearest neighbors +## Low-level interface: Nearest neighbors Nearest neighbors queries are computed by building a sparse hash grid defined as `hash_grid`. The grid is created by specifying a cell size for the @@ -218,7 +411,7 @@ find_neighbors(grid, neighbors, pt, max_dist); // find neighbors by pos find_neighbors(grid, neighbors, id, max_dist); // find neighbors by id ``` -## Element conversions and grouping +## Low-level interface: Element conversions and grouping Yocto/Shape support conversion between shape elements. Use `quads_to_triangles(quads)` to convert quads to triangles and @@ -301,13 +494,18 @@ primitive only. Use `merge_triangles_and_quads(triangles, quads, force_triangles to merge elements in-place. The algorithms will output quads if present or triangles if not unless `force_triangles` is used. -## Shape subdivision +## Low-level interface: Shape subdivision Yocto/Shape defines functions to subdivide shape elements linearly, in order to obtain higher shape resolution, for example before applying displacement mapping. All functions will split all shape elements, regardless of their size. This ensures that meshes have no cracks. -Use `subdivide_lines(lines, vert, level)` for lines, +Use `subdivide_lines(lines, vert)` for lines, +`subdivide_triangles(triangles, vert)` for triangles, +`subdivide_quads(quads, vert)` for quads, and +`subdivide_bezier(beziers, vert)` for Bezier segments. +Alternatively, shape elemens can be subdivideede multiple times, +as a convenience, with `subdivide_lines(lines, vert, level)` for lines, `subdivide_triangles(triangles, vert, level)` for triangles, `subdivide_quads(quads, vert, level)` for quads, and `subdivide_bezier(beziers, vert, level)` for Bezier segments. @@ -342,7 +540,7 @@ auto texcoords = vector{...}; auto [stquads, stexcoords] = subdivide_catmullclark(tquads, texcoords, 2, true); ``` -## Shape sampling +## Low-level interface: Shape sampling Yocto/Shape supports sampling meshes uniformly. All sampling require to first compute the shape CDF and then use it to sample the shape. For each shape type, @@ -382,7 +580,7 @@ sample_triangles(sampled_positions, sampled_normals, sampled_texcoords, triangles, positions, normals, texcoords, npoints); ``` -## Procedural shapes +## Low-level interface: Procedural shapes Yocto/Shape defines several procedural shapes used for both testing and to quickly create shapes for procedural scenes. Procedural shapes take as diff --git a/docs/yocto/yocto_shapeio.md b/docs/yocto/yocto_shapeio.md new file mode 100644 index 000000000..b7f495819 --- /dev/null +++ b/docs/yocto/yocto_shapeio.md @@ -0,0 +1,36 @@ +# Yocto/ShapeIO: Shape serialization + +Yocto/ShapeIO supports loading and saving shapes from Ply, Obj, Stl. +Yocto/ShapeIO is implemented in `yocto_shapeio.h` and `yocto_shapeio.cpp`. + +## Shape serialization + +Yocto/ShapeIO supports reading and writing shapes. +Use `load_shape(filename, shape)` or `shape = load_shape(filename)` +to load shapes, and `save_shape(filename, shape)` to save it. +Shapes are stored as `shape_data` structs defined in [Yocto/Shape](yocto_shape.md). +Upon errors, an `io_error` is thrown from all IO functions. +See [Yocto/CommonIO](yocto_commonio.md) for discussion on error handling +and use without exceptions. + +```cpp +auto shape = shape_data{}; +load_shape("input_file.ply", shape); // load shape +save_shape("output_file.ply", shape); // save shape +auto shape1 = load_shape("input_file.ply"); // alternative load +``` + +Yocto/ShapeIO supports reading and writing face-varying shapes. +Use `load_fvshape(filename, shape)` or `shape = load_fvshape(filename)` +to load shapes, and `save_shape(filename, shape)` to save it. +Shapes are stored as `fvshape_data` structs defined in [Yocto/Shape](yocto_shape.md). +Upon errors, an `io_error` is thrown from all IO functions. +See [Yocto/CommonIO](yocto_commonio.md) for discussion on error handling +and use without exceptions. + +```cpp +auto shape = fvshape_data{}; +load_fvshape("input_file.obj", shape); // load shape +save_fvshape("output_file.obj", shape); // save shape +auto shape1 = load_fvshape("input_file.obj"); // alternative load +``` diff --git a/libs/yocto/yocto_cli.cpp b/libs/yocto/yocto_cli.cpp index 8389c0144..6293dcab9 100644 --- a/libs/yocto/yocto_cli.cpp +++ b/libs/yocto/yocto_cli.cpp @@ -1028,18 +1028,4 @@ bool parse_cli(cli_state& cli, const vector& args, string& error) { } } -void parse_cli_and_handle_errors(cli_state& cli, const vector& args) { - try { - parse_cli(cli, args); - } catch (cli_error& error) { - print_info("error: " + string{error.what()}); - print_info(""); - print_info(get_usage(cli)); - exit(1); - } catch (cli_help& help) { - print_info(get_usage(cli)); - exit(0); - } -} - } // namespace yocto diff --git a/libs/yocto/yocto_cli.h b/libs/yocto/yocto_cli.h index f773eee33..074376cb8 100644 --- a/libs/yocto/yocto_cli.h +++ b/libs/yocto/yocto_cli.h @@ -138,8 +138,6 @@ cli_state make_cli(const string& cmd, const string& usage); void parse_cli(cli_state& cli, const vector& args); // parse arguments, checks for errors bool parse_cli(cli_state& cli, const vector& args, string& error); -// parse arguments, checks for errors, and exits on error or help -void parse_cli_and_handle_errors(cli_state& cli, const vector& args); // a convenience function that packs args to strings vector make_cli_args(int argc, const char** argv); diff --git a/libs/yocto/yocto_commonio.h b/libs/yocto/yocto_commonio.h index e07410e2b..1bff5a7ea 100644 --- a/libs/yocto/yocto_commonio.h +++ b/libs/yocto/yocto_commonio.h @@ -3,8 +3,8 @@ // // Yocto/CommonIO is a collection of utilities used in writing IO functionality, // including file IO, Json IO, and path manipulation. -// Yocto/CLI is implemented in `yocto_commonio.h` and `yocto_commonio.cpp`, and -// depends on `json.hpp` for Json serialization and number printing, and +// Yocto/CommonIO is implemented in `yocto_commonio.h` and `yocto_commonio.cpp`, +// and depends on `json.hpp` for Json serialization and number printing, and // `fast_float.h` for number parsing. // diff --git a/mkdocs.yml b/mkdocs.yml index ccebedf96..a745e5ea0 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -44,8 +44,11 @@ nav: - Mesh processing: yocto/yocto_mesh.md - Scene representation: yocto/yocto_scene.md - Ray-scene intersection: yocto/yocto_bvh.md - - Model serialization: yocto/yocto_modelio.md - - Scene serialization: yocto/yocto_sceneio.md - Path tracing: yocto/yocto_trace.md - Command-line utilities: yocto/yocto_cli.md + - Image serialization: yocto/yocto_imageio.md + - Shape serialization: yocto/yocto_shapeio.md + - Scene serialization: yocto/yocto_sceneio.md + - Json and serialization utilities: yocto/yocto_commonio.md + - Model serialization: yocto/yocto_modelio.md - Concurrency utilities: yocto/yocto_parallel.md diff --git a/readme.md b/readme.md index 33eecfcae..80f8cf17b 100644 --- a/readme.md +++ b/readme.md @@ -24,26 +24,28 @@ See each header file for documentation. functions, bsdf lobes, transmittance lobes, phase functions - `yocto/yocto_image.{h,cpp}`: simple image data type, image resizing, tonemapping, color correction, procedural images, procedural sun-sky -- `yocto/yocto_shape.{h,cpp}`: utilities for manipulating - triangle meshes, quads meshes and line sets, computation of normals and - tangents, linear and Catmull-Clark subdivision, procedural shapes generation, - ray intersection and closest point queries +- `yocto/yocto_shape.{h,cpp}`: simple shape data structure, utilities + for manipulating triangle meshes, quads meshes and line sets, computation of + normals and tangents, linear and Catmull-Clark subdivision, + procedural shapes generation, ray intersection and closest point queries - `yocto/yocto_mesh.{h,cpp}`: computational geometry utilities for triangle meshes, mesh geodesic, mesh cutting +- `yocto/yocto_scene.{h,cpp}`: scene representation and properties + evaluation - `yocto/yocto_bvh.{h,cpp}`: ray intersection and closest point queries of triangle meshes, quads meshes, line sets and instances scenes using a two-level bounding volume hierarchy -- `yocto/yocto_scene.{h,cpp}`: scene representation and properties - evaluation -- `yocto/yocto_sceneio.{h,cpp}`: image serialization, - shape serialization, scene serialization, text and binary serialization, - path helpers - `yocto/yocto_trace.{h,cpp}`: path tracing of surfaces and hairs supporting area and environment illumination, microfacet GGX and subsurface scattering, multiple importance sampling -- `yocto/yocto_modelio.{h,cpp}`: parsing and writing for Ply, Obj, - Stl, Pbrt formats -- `yocto/yocto_cli.h`: printing utilities, command line parsing +- `yocto/yocto_cli.{h,cpp}`: printing utilities and command line parsing +- `yocto/yocto_imageio.{h,cpp}`: image serialization +- `yocto/yocto_shapeio.{h,cpp}`: shape serialization +- `yocto/yocto_sceneio.{h,cpp}`: scene serialization +- `yocto/yocto_commonio.{h,cpp}`: common IO operations, path helpers, + text and binary serialization, Json data model, Json serialization +- `yocto/yocto_modelio.{h,cpp}`: low-level parsing and writing for + Ply, Obj, Stl, Pbrt formats - `yocto/yocto_parallel.h`: concurrency utilities You can see Yocto/GL in action in the following applications written to @@ -67,49 +69,41 @@ included in the [project site](https://xelatihy.github.io/yocto-gl/). Yocto/GL follows a "data-oriented programming model" that makes data explicit. Data is stored in simple structs and accessed with free functions or directly. All data is public, so we make no attempt at encapsulation. -Most objects is Yocto/GL have value semantic, while large data structures -use reference semantic with strict ownership. This means that everything -can be trivially serialized and there is no need for memory management. - We do this since this makes Yocto/GL easier to extend and quicker to learn, with a more explicit data flow that is easier when writing parallel code. Since Yocto/GL is mainly used for research and teaching, explicit data is both more hackable and easier to understand. +Nearly all objects in Yocto/GL have value semantic. This means that everything +can be trivially copied and serialized and there is no need for memory management. +While this has the drawback of potentially introducing spurious copies, it does +have the benefit of ensuring that no memory corruption can occur, which +turned out was a major issue for novice C++ users, even in a very small +library like this one. + In terms of code style we prefer a functional approach rather than an object oriented one, favoring free functions to class methods. All functions -and data are defined in sibling namespaces contained in the `yocto` namespace -so libraries can call all others, but have to do so explicitly. +and data are defined in the `yocto` namespace so libraries can call each others +trivially. The use of templates in Yocto was the reason for many refactoring, going from no template to heavy template use. At this point, Yocto uses some templates for readability. In the future, we will increase the use of templates in math code, while keeping many APIs explicitly typed. -We do not use exception for error reporting, but only to report "programmers" -errors. For example, IO operations use boolean flags and error strings for -human readable errors, while exceptions are used when preconditions or -post conditions are violated in functions. - -After several refactoring, we settled on a value-based approach, since the use -of pointers and reference semantics was hard for many of our users. While -this has the drawback of potentially introducing spurious copies, it does -have th benefit of ensuring that no memory corruption can occur, which -turned out was a major issue for novice C++ users, even in a very small -library like this one. - -## Credits - -Main contributors: +We support both exceptions and return codes for error reporting. For example, +we have two versions of many IO operations that either use boolean flags and +error strings for human readable errors, or throw exceptions with the same +error string. Internally exceptions are used, since they provide cleaner +code and where in fact faster ion low-level parsers. At the moment, +exceptions are only used in IO functions and to report "programmer" +errors, namely when preconditions or post conditions are violated in functions. -- Fabio Pellacini (lead developer): [web](http://pellacini.di.uniroma1.it), [github](https://github.com/xelatihy) -- Edoardo Carra: [github](https://github.com/edoardocarra) -- Giacomo Nazzaro: [github](https://github.com/giacomonazzaro) +## License -This library includes code from the [PCG random number generator](http://www.pcg-random.org), -boost `hash_combine`, and public domain code from `github.com/sgorsten/linalg`, -`gist.github.com/badboy/6267743` and `github.com/nothings/stb_perlin.h`. -Other external libraries are included with their own license. +The library is released under the MIT license. We include various external +dependencies in the distribution that each have thir own license, compatible +with the chosen one. ## Compilation @@ -117,22 +111,20 @@ This library requires a C++17 compiler and is know to compiled on OsX (Xcode >= 11), Windows (MSVC 2019) and Linux (gcc >= 9, clang >= 9). You can build the example applications using CMake with -`mkdir build; cd build; cmake ..; cmake --build .` +`mkdir build; cd build; cmake ..; cmake --build` -Yocto/GL depends on `stb_image.h`, `stb_image_write.h`, `stb_image_resize.h` and -`tinyexr.h` for image loading, saving and resizing, `cgltf.h` and `json.hpp` -for glTF and JSON support, and `filesystem.hpp` to support C++17 filesystem API -when missing. All dependencies are included in the distribution. +Yocto/GL required dependencies are included in the distribution and do not +need to be installed separately. -Yocto/GL optionally supports building OpenGL demos, which are handled by including -glad, GLFW, ImGui as dependencies in apps. OpenGL support might eventually -become part of the Yocto/GL libraries. OpenGL support is enabled by defining -the cmake option `YOCTO_OPENGL` and contained in the `yocto_gui` library. +Yocto/GL optionally supports building OpenGL demos. OpenGL support is enabled +by defining the cmake option `YOCTO_OPENGL` and contained in the `yocto_gui` +library. OpenGL dependencies are included in this repo. Yocto/GL optionally supports the use of Intel's Embree for ray casting. See the main CMake file for how to link to it. Embree support is enabled by -defining the cmake option `YOCTO_EMBREE`. +defining the cmake option `YOCTO_EMBREE`. Embree needs to be installed separately. Yocto/GL optionally supports the use of Intel's Open Image Denoise for denoising. See the main CMake file for how to link to it. Open Image Denoise support -is enabled by defining the cmake option `YOCTO_DENOISE`. +is enabled by defining the cmake option `YOCTO_DENOISE`. +OIDN needs to be installed separately.