Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/3176 json #3191

Merged
merged 40 commits into from
May 24, 2023
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
221dc33
Merge branch 'develop' of https://github.com/stan-dev/stan into develop
mitzimorris Mar 16, 2023
288bcca
Merge branch 'develop' of https://github.com/stan-dev/stan into develop
mitzimorris Mar 25, 2023
7433bb4
Merge branch 'develop' of https://github.com/stan-dev/stan into develop
mitzimorris Apr 5, 2023
339ea9f
Merge branch 'develop' of https://github.com/stan-dev/stan into develop
mitzimorris Apr 6, 2023
29afdfc
adding json writer, remove get_stream from unique_stream_writer
mitzimorris Apr 7, 2023
3b53e40
unit test cleanup
mitzimorris Apr 7, 2023
fb6c1c9
fix unit tests calling unique_stream_writer.get_stream
mitzimorris Apr 8, 2023
42c42a6
reverting changes to unique_stream_writer
mitzimorris Apr 9, 2023
0e1313a
bring code into line w/ multi-path branch
mitzimorris May 16, 2023
3f862d8
Merge commit '2338b544ed7bed8cef066ef3aa2f147a56f80bca' into HEAD
yashikno May 16, 2023
c48925c
[Jenkins] auto-formatting by clang-format version 10.0.0-4ubuntu1
stan-buildbot May 16, 2023
68f22e5
Merge branch 'feature/3176-json' of https://github.com/stan-dev/stan …
mitzimorris May 16, 2023
9791274
adding in changes to callbacks from branch multi-path
mitzimorris May 16, 2023
5739531
changes per code review
mitzimorris May 16, 2023
40da8b9
changes per code review
mitzimorris May 17, 2023
fea6b02
[Jenkins] auto-formatting by clang-format version 10.0.0-4ubuntu1
stan-buildbot May 17, 2023
aa83b0d
changes per code review
mitzimorris May 17, 2023
b40f535
Merge branch 'feature/3176-json' of https://github.com/stan-dev/stan …
mitzimorris May 17, 2023
ef056e2
adding unique_stream_writer tests from branch multi-path
mitzimorris May 17, 2023
73415a9
doc comments
mitzimorris May 18, 2023
39b73f7
unique stream writer TEST (was impl)
mitzimorris May 18, 2023
46e1085
fix write Inf, NaN
mitzimorris May 18, 2023
82e6bbc
Merge commit '1b126f5c8da9d374d0bee934779a6600112ed08c' into HEAD
yashikno May 18, 2023
7571287
[Jenkins] auto-formatting by clang-format version 10.0.0-4ubuntu1
stan-buildbot May 18, 2023
7db2517
Merge branch 'feature/3176-json' of https://github.com/stan-dev/stan …
mitzimorris May 18, 2023
24fa869
generalized json_writer, more unit tests
mitzimorris May 18, 2023
579f03e
[Jenkins] auto-formatting by clang-format version 10.0.0-4ubuntu1
stan-buildbot May 18, 2023
b13018b
Merge branch 'feature/3176-json' of https://github.com/stan-dev/stan …
mitzimorris May 18, 2023
a26b246
more unit tests
mitzimorris May 18, 2023
b6a6144
[Jenkins] auto-formatting by clang-format version 10.0.0-4ubuntu1
stan-buildbot May 18, 2023
8a0e135
changes per code review
mitzimorris May 19, 2023
83ac759
Merge branch 'feature/3176-json' of https://github.com/stan-dev/stan …
mitzimorris May 19, 2023
7baef99
allow nullptr; check everywhere
mitzimorris May 19, 2023
4411653
unit tests
mitzimorris May 19, 2023
0d9c9c3
Merge commit '0ae7279f63be757f5bb3b753b3f8ee232a504bc0' into HEAD
yashikno May 19, 2023
70ec6e8
[Jenkins] auto-formatting by clang-format version 10.0.0-4ubuntu1
stan-buildbot May 19, 2023
3e848f9
more unit tests
mitzimorris May 23, 2023
facc14c
Merge branch 'feature/3176-json' of https://github.com/stan-dev/stan …
mitzimorris May 23, 2023
2b3bc01
Merge commit 'fd20bd115098a28813beaf2f92d075130df5b248' into HEAD
yashikno May 23, 2023
aea439c
[Jenkins] auto-formatting by clang-format version 10.0.0-4ubuntu1
stan-buildbot May 23, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
362 changes: 362 additions & 0 deletions src/stan/callbacks/json_writer.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,362 @@
#ifndef STAN_CALLBACKS_JSON_WRITER_HPP
#define STAN_CALLBACKS_JSON_WRITER_HPP

#include <stan/math/prim/fun/Eigen.hpp>
#include <ostream>
#include <string>
#include <vector>
#include <memory>
#include <cstring>

namespace stan {
namespace callbacks {

/**
* The `json_writer` callback is used output a single JSON object.
* A JSON object is a mapping from element names to values which can be
* either a scalar or array element, or a nested JSON object.
* Objects are output elementwise via `write` callbacks which send
* key, value pairs to the output stream. Because JSON format
* requires a comma between elements, the writer maintains
* internal state to determine whether or not to output the comma separator.
* The writer doesn't try to validate the object's internal structure
* or object completeness, only syntactic correctness.
*
* @tparam Stream A type with with a valid `operator<<(std::string)`
* @tparam Deleter A class with a valid `operator()` method for deleting the
* output stream
*/
template <typename Stream, typename Deleter = std::default_delete<Stream>>
class json_writer {
private:
// Output stream
std::unique_ptr<Stream, Deleter> output_{nullptr};
// Whether or not the record's current object needs a comma separator
bool record_element_needs_comma_ = false;
// Depth of records (used to determine whether or not to print comma
// separator)
int record_depth_ = 0;
// Whether or not the record's parent object needs a comma separator
bool record_needs_comma_;

/**
* Writes a comma separator for the record's parent object if needed.
*/
void write_record_comma_if_needed() {
if (record_depth_ > 0 && record_needs_comma_) {
*output_ << ",";
}
}

/**
* Determines whether a record's internal object requires a comma separator
*/
void write_sep() {
if (record_element_needs_comma_) {
*output_ << ", ";
} else {
record_element_needs_comma_ = true;
}
}

/**
* Process a string to escape special characters.
* Valid json strings cannot contain any of the special characters
* `'\\', '"', '/', '\b', '\f', '\n', '\r', '\t', '\v', '\a', '\0'`.
* In order to print these characters, they must be escaped.
* @param value The string to process.
* @return The processed string.
*/
std::string process_string(const std::string& value) {
static constexpr std::array<char, 11> chars_to_escape
= {'\\', '"', '/', '\b', '\f', '\n', '\r', '\t', '\v', '\a', '\0'};
static constexpr std::array<const char*, 11> chars_to_replace
= {"\\\\", "\\\"", "\\/", "\\b", "\\f", "\\n",
"\\r", "\\t", "\\v", "\\a", "\\0"};
// Replacing every value leads to 2x the size
std::string new_value(value.size() * 2, 'x');
std::size_t pos = 0;
std::size_t count = 0;
std::size_t prev_pos = 0;
while ((pos = value.find_first_of(chars_to_escape.data(), pos, 10))
!= std::string::npos) {
for (int i = prev_pos; i < pos; ++i) {
new_value[i + count] = value[i];
}
int idx
= strchr(chars_to_escape.data(), value[pos]) - chars_to_escape.data();
new_value[pos + count] = chars_to_replace[idx][0];
new_value[pos + count + 1] = chars_to_replace[idx][1];
pos += 1;
count++;
prev_pos = pos;
}
for (int i = prev_pos; i < value.size(); ++i) {
new_value[i + count] = value[i];
}
// Shrink any unused space
new_value.resize(value.size() + count);
return new_value;
}

/**
* Writes key plus colon for key-value pair.
*
* @param[in] key member name.
*/
void write_key(const std::string& key) { *output_ << "\"" << key << "\" : "; }
/**
* Writes a set of comma separated strings.
* Strings are cleaned to escape special characters.
*
* @param[in] v Values in a std::vector
*/
void write_vector(const std::vector<std::string>& v) {
if (v.empty()) {
return;
}
*output_ << "[ ";
auto last = v.end();
--last;
for (auto it = v.begin(); it != last; ++it) {
*output_ << process_string(*it) << ", ";
}
*output_ << v.back() << " ]";
}

/**
* Writes a set of comma separated values.
*
* @param[in] v Values in a std::vector
*/
template <class T>
void write_vector(const std::vector<T>& v) {
if (v.empty()) {
return;
}
*output_ << "[ ";
auto last = v.end();
--last;
for (auto it = v.begin(); it != last; ++it) {
*output_ << *it << ", ";
}
*output_ << v.back() << " ]";
}

public:
json_writer() : output_(nullptr) {}
mitzimorris marked this conversation as resolved.
Show resolved Hide resolved
/**
* Constructs a json writer with an output stream.
*
* @param[in, out] output unique pointer to a type inheriting from
* `std::ostream`
*/
explicit json_writer(std::unique_ptr<Stream, Deleter>&& output)
: output_(std::move(output)) {
if (output_ == nullptr)
throw std::invalid_argument("writer cannot be null");
}

json_writer(json_writer& other) = delete;
json_writer(json_writer&& other) : output_(std::move(other.output_)) {}
mitzimorris marked this conversation as resolved.
Show resolved Hide resolved

~json_writer() {}

/**
* Writes "{", initial token of a JSON record.
*/
void begin_record() {
write_record_comma_if_needed();
*output_ << "{";
record_needs_comma_ = false;
record_depth_++;
}

/**
* Writes "\"key\" : {", initial token of a named JSON record.
* @param[in] key The name of the record.
*/
void begin_record(const std::string& key) {
write_record_comma_if_needed();
*output_ << "\"" << key << "\": {";
record_needs_comma_ = false;
record_depth_++;
}
/**
* Writes "}", final token of a JSON record.
*/
void end_record() {
*output_ << "}";
record_depth_--;
if (record_depth_ > 0) {
record_needs_comma_ = true;
}
record_element_needs_comma_ = false;
}

/**
* Write a key-value pair to the output stream with a value of null as the
* value.
* @param key Name of the value pair
*/
void write(const std::string& key) {
write_sep();
write_key(key);
*output_ << "null";
}

/**
* Write a key-value pair where the value is a string.
* @param key Name of the value pair
* @param value string to write.
*/
void write(const std::string& key, const std::string& value) {
std::string processsed_string = process_string(value);
write_sep();
write_key(key);
*output_ << "\"" << processsed_string << "\"";
}

/**
* Write a key-value pair where the value is a const char*.
* @param key Name of the value pair
* @param value pointer to chars to write.
*/
void write(const std::string& key, const char* value) {
std::string processsed_string = process_string(value);
write_sep();
write_key(key);
*output_ << "\"" << processsed_string << "\"";
}

/**
* Write a key-value pair where the value is a bool.
* @param key Name of the value pair
* @param value bool to write.
*/
void write(const std::string& key, bool value) {
write_sep();
write_key(key);
*output_ << (value ? "true" : "false");
}

/**
* Write a key-value pair where the value is an int.
* @param key Name of the value pair
* @param value int to write.
*/
void write(const std::string& key, int value) {
write_sep();
write_key(key);
*output_ << value;
}

/**
* Write a key-value pair where the value is an `std::size_t`.
* @param key Name of the value pair
* @param value `std::size_t` to write.
*/
void write(const std::string& key, std::size_t value) {
write_sep();
write_key(key);
*output_ << value;
}

/**
* Write a key-value pair where the value is a double.
* @param key Name of the value pair
* @param value double to write.
*/
void write(const std::string& key, double value) {
write_sep();
write_key(key);
*output_ << value;
}

/**
* Write a key-value pair where the value is a complex value.
* @param key Name of the value pair
* @param value complex value to write.
*/
void write(const std::string& key, const std::complex<double>& value) {
write_sep();
write_key(key);
*output_ << "\"" << key << "\" : [" << value.real() << ", " << value.imag()
<< "]";
}

/**
* Write a key-value pair where the value is a vector to be made a list.
* @param key Name of the value pair
* @param values vector to write.
*/
void write(const std::string& key, const std::vector<double>& values) {
write_sep();
write_key(key);
write_vector(values);
}

/**
* Write a key-value pair where the value is a vector of strings to be made a
* list.
* @param key Name of the value pair
* @param values vector of strings to write.
*/
void write(const std::string& key, const std::vector<std::string>& values) {
write_sep();
write_key(key);
write_vector(values);
}

/**
* Write a key-value pair where the value is an Eigen Matrix.
* @param key Name of the value pair
* @param mat Eigen Matrix to write.
*/
void write(const std::string& key, const Eigen::MatrixXd& mat) {
write_sep();
write_key(key);
Eigen::IOFormat json_format(Eigen::StreamPrecision, Eigen::DontAlignCols,
", ", ", ", "[", "]", "[", "]");
*output_ << mat.format(json_format);
}

/**
* Write a key-value pair where the value is an Eigen Vector.
* @param key Name of the value pair
* @param vec Eigen Vector to write.
*/
void write(const std::string& key, const Eigen::VectorXd& vec) {
write_sep();
write_key(key);
Eigen::IOFormat json_format(Eigen::StreamPrecision, Eigen::DontAlignCols,
", ", "", "", "", "[", "]");
*output_ << vec.transpose().format(json_format);
}

/**
* Write a key-value pair where the value is a Eigen RowVector.
* @param key Name of the value pair
* @param vec Eigen RowVector to write.
*/
void write(const std::string& key, const Eigen::RowVectorXd& vec) {
write_sep();
write_key(key);
Eigen::IOFormat json_format(Eigen::StreamPrecision, Eigen::DontAlignCols,
", ", "", "", "", "[", "]");
*output_ << vec.format(json_format);
}

/**
* Reset state
*/
void reset() {
WardBrian marked this conversation as resolved.
Show resolved Hide resolved
record_element_needs_comma_ = false;
record_needs_comma_ = false;
record_depth_ = 0;
}
};

} // namespace callbacks
} // namespace stan
#endif
Loading