As I said in previous cover letter: This patchset has multiple commits, but isn't really large. Most of the lines come from CBOR::Any, which only is a copy of JSON::Any. Each feature added has at least some tests: JSON::Any encodings, CBOR::Any (working as JSON::Any) and Time. Also, Time now can be formatted as the JSON Time representation. Karchnu (8): Write JSON::Any Add CBOR::Any. Time added for CBOR::Any and CBOR::Encoder. Time: allow to use String representations, as produced by #to_json. Time.new fixed: now provides a Time instance or raises. Add CBOR::Any tests. Add CBOR Time tests. Applying "crystal tool format" patches. spec/cbor_any.cr | 33 ++++ spec/cbor_time.cr | 15 ++ src/cbor/any.cr | 388 ++++++++++++++++++++++++++++++++++++++++++ src/cbor/encoder.cr | 15 ++ src/cbor/from_cbor.cr | 38 +++-- 5 files changed, 476 insertions(+), 13 deletions(-) create mode 100644 spec/cbor_any.cr create mode 100644 spec/cbor_time.cr create mode 100644 src/cbor/any.cr -- 2.30.2
crystal-cbor/patches/.build.yml: SUCCESS in 28s [Rewrite of CBOR::Any patchset (with better source format)][0] v2 from [~karchnu][1] [0]: https://lists.sr.ht/~arestifo/crystal-cbor/patches/23445 [1]: mailto:karchnu@karchnu.fr ✓ #531503 SUCCESS crystal-cbor/patches/.build.yml https://builds.sr.ht/~arestifo/job/531503
Copy & paste the following snippet into your terminal to import this patchset into git:
curl -s https://lists.sr.ht/~arestifo/crystal-cbor/patches/23445/mbox | git am -3Learn more about email & git
From: Karchnu <karchnu@karchnu.fr> --- src/cbor/encoder.cr | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/src/cbor/encoder.cr b/src/cbor/encoder.cr index 65e5e63..92678b4 100644 --- a/src/cbor/encoder.cr +++ b/src/cbor/encoder.cr @@ -1,3 +1,5 @@ +require "json" + class CBOR::Encoder def self.new(io : IO = IO::Memory.new) packer = new(io) @@ -8,6 +10,10 @@ class CBOR::Encoder def initialize(@io : IO = IO::Memory.new) end + def write(j : JSON::Any) + write j.raw + end + def write(value : Nil | Nil.class, use_undefined : Bool = false) write(use_undefined ? SimpleValue::Undefined : SimpleValue::Null) end -- 2.30.2
From: Karchnu <karchnu@karchnu.fr> --- src/cbor/any.cr | 392 ++++++++++++++++++++++++++++++++++++++++++++ src/cbor/encoder.cr | 5 + 2 files changed, 397 insertions(+) create mode 100644 src/cbor/any.cr diff --git a/src/cbor/any.cr b/src/cbor/any.cr new file mode 100644 index 0000000..f6a551b --- /dev/null +++ b/src/cbor/any.cr @@ -0,0 +1,392 @@ +# `CBOR::Any` is a convenient wrapper around all possible CBOR types (`CBOR::Any::Type`) +# and can be used for traversing dynamic or unknown CBOR structures. +# +# ``` +# require "json" +# +# obj = CBOR::Any.new JSON.parse(%({"access": [{"name": "mapping", "speed": "fast"}, {"name": "any", "speed": "slow"}]})) +# obj["access"][1]["name"].as_s # => "any" +# obj["access"][1]["speed"].as_s # => "slow" +# +# # through a cbor buffer +# hash = {"access" => [{"name" => "mapping", "speed" => "fast"}, {"name" => "any", "speed" => "slow"}]} +# obj2 = CBOR::Any.new hash.to_cbor +# obj2["access"][1]["name"].as_s # => "any" +# obj2["access"][1]["speed"].as_s # => "slow" +# ``` +# +# Note that methods used to traverse a CBOR structure, `#[]` and `#[]?`, +# always return a `CBOR::Any` to allow further traversal. To convert them to `String`, +# `Int32`, etc., use the `as_` methods, such as `#as_s`, `#as_i`, which perform +# a type check against the raw underlying value. This means that invoking `#as_s` +# when the underlying value is not a String will raise: the value won't automatically +# be converted (parsed) to a `String`. +struct CBOR::Any + + # All possible CBOR types. + alias Type = Nil | Bool | String | Bytes | + Int8 | UInt8 | Int16 | UInt16 | Int32 | UInt32 | Int64 | UInt64 | Int128 | + Float32 | Float64 | + Array(Any) | + Hash(String, Any) | Hash(Any, Any) + + # Reads a `CBOR::Any` from a JSON::Any structure. + def self.new(json : JSON::Any) + new json.raw + end + + def self.new(value) + case value + when Nil + new nil + when Bool + new value + when Int64 + new value + when Float64 + new value + when String + new value + + when Array(Type) + ary = [] of CBOR::Any + value.each do |v| + ary << new(v) + end + new ary + + when Array(JSON::Any) + ary = [] of CBOR::Any + value.each do |v| + ary << new(v) + end + new ary + + when Array(CBOR::Type) + ary = [] of CBOR::Any + value.each do |v| + ary << new(v) + end + new ary + + when Hash(String, JSON::Any) + hash = {} of String => CBOR::Any + value.each do |key, v| + hash[key] = new(v) + end + new hash + + when Hash(CBOR::Type, CBOR::Type) + hash = {} of CBOR::Any => CBOR::Any + value.each do |key, v| + hash[new(key)] = new(v) + end + new hash + + when Hash(String, Type) + hash = {} of String => CBOR::Any + value.each do |key, v| + hash[key] = new(v) + end + new hash + else + raise "Unknown value type: #{value.class}" + end + end + + # Reads a `CBOR::Any` from a Decoder. + def self.new(decoder : CBOR::Decoder) + new decoder.read_value + end + + # Reads a `CBOR::Any` from a buffer. + def self.new(input : Slice(UInt8)) + new CBOR::Decoder.new(input) + end + + # Returns the raw underlying value. + getter raw : Type + + # Creates a `CBOR::Any` that wraps the given value. + def initialize(@raw : Type) + end + + # Assumes the underlying value is an `Array` or `Hash` and returns its size. + # Raises if the underlying value is not an `Array` or `Hash`. + def size : Int + case object = @raw + when Array + object.size + when Hash + object.size + else + raise "Expected Array or Hash for #size, not #{object.class}" + end + end + + # Assumes the underlying value is an `Array` and returns the element + # at the given index. + # Raises if the underlying value is not an `Array`. + def [](index : Int) : CBOR::Any + case object = @raw + when Array + object[index] + else + raise "Expected Array for #[](index : Int), not #{object.class}" + end + end + + # Assumes the underlying value is an `Array` and returns the element + # at the given index, or `nil` if out of bounds. + # Raises if the underlying value is not an `Array`. + def []?(index : Int) : CBOR::Any? + case object = @raw + when Array + object[index]? + else + raise "Expected Array for #[]?(index : Int), not #{object.class}" + end + end + + # Assumes the underlying value is a `Hash` and returns the element + # with the given key. + # Raises if the underlying value is not a `Hash`. + def [](key : String) : CBOR::Any + case object = @raw + when Hash + object[key] + else + raise "Expected Hash for #[](key : String), not #{object.class}" + end + end + + # Assumes the underlying value is a `Hash` and returns the element + # with the given key, or `nil` if the key is not present. + # Raises if the underlying value is not a `Hash`. + def []?(key : String) : CBOR::Any? + case object = @raw + when Hash + object[key]? + else + raise "Expected Hash for #[]?(key : String), not #{object.class}" + end + end + + # Traverses the depth of a structure and returns the value. + # Returns `nil` if not found. + def dig?(key : String | Int, *subkeys) + if value = self[key]? + value.dig?(*subkeys) + end + end + + # :nodoc: + def dig?(key : String | Int) + case @raw + when Hash, Array + self[key]? + else + nil + end + end + + # Traverses the depth of a structure and returns the value, otherwise raises. + def dig(key : String | Int, *subkeys) + if (value = self[key]) && value.responds_to?(:dig) + return value.dig(*subkeys) + end + raise "CBOR::Any value not diggable for key: #{key.inspect}" + end + + # :nodoc: + def dig(key : String | Int) + self[key] + end + + # Checks that the underlying value is `Nil`, and returns `nil`. + # Raises otherwise. + def as_nil : Nil + @raw.as(Nil) + end + + # Checks that the underlying value is `Bool`, and returns its value. + # Raises otherwise. + def as_bool : Bool + @raw.as(Bool) + end + + # Checks that the underlying value is `Bool`, and returns its value. + # Returns `nil` otherwise. + def as_bool? : Bool? + as_bool if @raw.is_a?(Bool) + end + + # Checks that the underlying value is `Int`, and returns its value as an `Int32`. + # Raises otherwise. + def as_i : Int32 + @raw.as(Int).to_i + end + + # Checks that the underlying value is `Int`, and returns its value as an `Int32`. + # Returns `nil` otherwise. + def as_i? : Int32? + as_i if @raw.is_a?(Int) + end + + # Checks that the underlying value is `Int`, and returns its value as an `Int64`. + # Raises otherwise. + def as_i64 : Int64 + @raw.as(Int).to_i64 + end + + # Checks that the underlying value is `Int`, and returns its value as an `Int64`. + # Returns `nil` otherwise. + def as_i64? : Int64? + as_i64 if @raw.is_a?(Int64) + end + + # Checks that the underlying value is `Float`, and returns its value as an `Float64`. + # Raises otherwise. + def as_f : Float64 + @raw.as(Float64) + end + + # Checks that the underlying value is `Float`, and returns its value as an `Float64`. + # Returns `nil` otherwise. + def as_f? : Float64? + @raw.as?(Float64) + end + + # Checks that the underlying value is `Float`, and returns its value as an `Float32`. + # Raises otherwise. + def as_f32 : Float32 + @raw.as(Float).to_f32 + end + + # Checks that the underlying value is `Float`, and returns its value as an `Float32`. + # Returns `nil` otherwise. + def as_f32? : Float32? + as_f32 if @raw.is_a?(Float) + end + + # Checks that the underlying value is `String`, and returns its value. + # Raises otherwise. + def as_s : String + @raw.as(String) + end + + # Checks that the underlying value is `String`, and returns its value. + # Returns `nil` otherwise. + def as_s? : String? + as_s if @raw.is_a?(String) + end + + # Checks that the underlying value is `Array`, and returns its value. + # Raises otherwise. + def as_a : Array(Any) + @raw.as(Array) + end + + # Checks that the underlying value is `Array`, and returns its value. + # Returns `nil` otherwise. + def as_a? : Array(Any)? + as_a if @raw.is_a?(Array) + end + + # Checks that the underlying value is `Hash`, and returns its value. + # Raises otherwise. + def as_h : Hash(String, Any) + @raw.as(Hash) + end + + # Checks that the underlying value is `Hash`, and returns its value. + # Returns `nil` otherwise. + def as_h? : Hash(String, Any)? + as_h if @raw.is_a?(Hash) + end + + # :nodoc: + def inspect(io : IO) : Nil + @raw.inspect(io) + end + + # :nodoc: + def to_s(io : IO) : Nil + @raw.to_s(io) + end + + # :nodoc: + def pretty_print(pp) + @raw.pretty_print(pp) + end + + # Returns `true` if both `self` and *other*'s raw object are equal. + def ==(other : CBOR::Any) + raw == other.raw + end + + # Returns `true` if the raw object is equal to *other*. + def ==(other) + raw == other + end + + # See `Object#hash(hasher)` + def_hash raw + + # :nodoc: + def to_json(json : JSON::Builder) + raw.to_json(json) + end + + def to_yaml(yaml : YAML::Nodes::Builder) + raw.to_yaml(yaml) + end + + # Returns a new CBOR::Any instance with the `raw` value `dup`ed. + def dup + Any.new(raw.dup) + end + + # Returns a new CBOR::Any instance with the `raw` value `clone`ed. + def clone + Any.new(raw.clone) + end +end + +class Object + def ===(other : CBOR::Any) + self === other.raw + end +end + +struct Value + def ==(other : CBOR::Any) + self == other.raw + end +end + +class Reference + def ==(other : CBOR::Any) + self == other.raw + end +end + +class Array + def ==(other : CBOR::Any) + self == other.raw + end +end + +class Hash + def ==(other : CBOR::Any) + self == other.raw + end +end + +class Regex + def ===(other : CBOR::Any) + value = self === other.raw + $~ = $~ + value + end +end diff --git a/src/cbor/encoder.cr b/src/cbor/encoder.cr index 92678b4..b355a59 100644 --- a/src/cbor/encoder.cr +++ b/src/cbor/encoder.cr @@ -10,6 +10,11 @@ class CBOR::Encoder def initialize(@io : IO = IO::Memory.new) end + def write(cbor : CBOR::Any) + # Test each possible value of CBOR::Any + write cbor.raw + end + def write(j : JSON::Any) write j.raw end -- 2.30.2
From: Karchnu <karchnu@karchnu.fr> --- src/cbor/any.cr | 5 ++++- src/cbor/encoder.cr | 7 ++++++- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/src/cbor/any.cr b/src/cbor/any.cr index f6a551b..36c2db9 100644 --- a/src/cbor/any.cr +++ b/src/cbor/any.cr @@ -28,7 +28,8 @@ struct CBOR::Any Int8 | UInt8 | Int16 | UInt16 | Int32 | UInt32 | Int64 | UInt64 | Int128 | Float32 | Float64 | Array(Any) | - Hash(String, Any) | Hash(Any, Any) + Hash(String, Any) | Hash(Any, Any) | + Time # Reads a `CBOR::Any` from a JSON::Any structure. def self.new(json : JSON::Any) @@ -47,6 +48,8 @@ struct CBOR::Any new value when String new value + when Time + new value when Array(Type) ary = [] of CBOR::Any diff --git a/src/cbor/encoder.cr b/src/cbor/encoder.cr index b355a59..67bf1c4 100644 --- a/src/cbor/encoder.cr +++ b/src/cbor/encoder.cr @@ -11,7 +11,6 @@ class CBOR::Encoder end def write(cbor : CBOR::Any) - # Test each possible value of CBOR::Any write cbor.raw end @@ -45,6 +44,12 @@ class CBOR::Encoder write(value.to_s) end + def write(value : Time) + write(CBOR::Tag::RFC3339Time) + write(value.to_rfc3339) + end + + def write(value : Float32 | Float64) case value when Float32 -- 2.30.2
From: Karchnu <karchnu@karchnu.fr> --- src/cbor/from_cbor.cr | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/src/cbor/from_cbor.cr b/src/cbor/from_cbor.cr index e2610e7..4979fa8 100644 --- a/src/cbor/from_cbor.cr +++ b/src/cbor/from_cbor.cr @@ -134,18 +134,24 @@ end # # [1]: https://tools.ietf.org/html/rfc7049#section-2.4.1 def Time.new(decoder : CBOR::Decoder) - case tag = decoder.read_tag - when CBOR::Tag::RFC3339Time + # In case Time is formatted as a JSON#to_json String. + case decoder.current_token + when CBOR::Token::StringT Time::Format::RFC_3339.parse(decoder.read_string) - when CBOR::Tag::EpochTime - case num = decoder.read_num - when Int - Time.unix(num) - when Float - Time.unix_ms((BigFloat.new(num) * 1_000).to_u64) - end else - raise CBOR::ParseError.new("Expected tag to have value 0 or 1, got #{tag.value}") + case tag = decoder.read_tag + when CBOR::Tag::RFC3339Time + Time::Format::RFC_3339.parse(decoder.read_string) + when CBOR::Tag::EpochTime + case num = decoder.read_num + when Int + Time.unix(num) + when Float + Time.unix_ms((BigFloat.new(num) * 1_000).to_u64) + end + else + raise CBOR::ParseError.new("Expected tag to have value 0 or 1, got #{tag.value}") + end end end -- 2.30.2
From: Karchnu <karchnu@karchnu.fr> --- src/cbor/from_cbor.cr | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/src/cbor/from_cbor.cr b/src/cbor/from_cbor.cr index 4979fa8..08f1be3 100644 --- a/src/cbor/from_cbor.cr +++ b/src/cbor/from_cbor.cr @@ -133,9 +133,9 @@ end # specified by [Section 2.4.1 of RFC 7049][1]. # # [1]: https://tools.ietf.org/html/rfc7049#section-2.4.1 -def Time.new(decoder : CBOR::Decoder) +def Time.new(decoder : CBOR::Decoder) : Time # In case Time is formatted as a JSON#to_json String. - case decoder.current_token + value = case decoder.current_token when CBOR::Token::StringT Time::Format::RFC_3339.parse(decoder.read_string) else @@ -153,6 +153,12 @@ def Time.new(decoder : CBOR::Decoder) raise CBOR::ParseError.new("Expected tag to have value 0 or 1, got #{tag.value}") end end + + unless value + raise CBOR::ParseError.new("could not parse time representation") + end + + value end # Reads the CBOR value as a BigInt. -- 2.30.2
From: Karchnu <karchnu@karchnu.fr> --- spec/cbor_any.cr | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 spec/cbor_any.cr diff --git a/spec/cbor_any.cr b/spec/cbor_any.cr new file mode 100644 index 0000000..fe43a29 --- /dev/null +++ b/spec/cbor_any.cr @@ -0,0 +1,33 @@ +require "./spec_helper" + +describe CBOR do + describe "CBOR::Any" do + it "from JSON::Any - 1" do + h_any = {"a" => "b", "c" => "d", "e" => true, "f" => 10} + json_any = JSON.parse h_any.to_json + cbor_any = CBOR::Any.new json_any + cbor_any.to_cbor.hexstring.should eq "a461616162616361646165f561660a" + end + + it "from JSON::Any - 2" do + h_any = {"a" => "b", "c" => "d", "e" => true, "f" => 10} + json_any = JSON.parse h_any.to_json + cbor_any = CBOR::Any.new json_any + cbor_any["f"].should eq 10 + end + + it "from array" do + array = CBOR::Any.from_cbor [ "a", "b", "c", "d" ].to_cbor + array.to_cbor.hexstring.should eq "846161616261636164" + end + + it "from hash" do + h_cany = Hash(String | Int32, String | Int32).new + h_cany["name"] = "Alice" + h_cany["age"] = 30 + h_cany["size"] = 160 + cbor_any_hash = CBOR::Any.from_cbor h_cany.to_cbor + cbor_any_hash.to_cbor.hexstring.should eq "a3646e616d6565416c69636563616765181e6473697a6518a0" + end + end +end -- 2.30.2
From: Karchnu <karchnu@karchnu.fr> --- spec/cbor_time.cr | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 spec/cbor_time.cr diff --git a/spec/cbor_time.cr b/spec/cbor_time.cr new file mode 100644 index 0000000..a6e99c0 --- /dev/null +++ b/spec/cbor_time.cr @@ -0,0 +1,15 @@ +require "./spec_helper" + +describe CBOR do + describe "Time" do + it "Time#to_cbor" do + time = Time.utc(2016, 2, 15, 10, 20, 30) + time.to_cbor.hexstring.should eq "c074323031362d30322d31355431303a32303a33305a" + end + + it "Time#from_cbor" do + time = Time.from_cbor Time.utc(2016, 2, 15, 10, 20, 30).to_cbor + time.to_cbor.hexstring.should eq "c074323031362d30322d31355431303a32303a33305a" + end + end +end -- 2.30.2
From: Karchnu <karchnu@karchnu.fr> --- spec/cbor_any.cr | 4 ++-- src/cbor/any.cr | 7 ------- src/cbor/encoder.cr | 1 - src/cbor/from_cbor.cr | 34 +++++++++++++++++----------------- 4 files changed, 19 insertions(+), 27 deletions(-) diff --git a/spec/cbor_any.cr b/spec/cbor_any.cr index fe43a29..b5b6097 100644 --- a/spec/cbor_any.cr +++ b/spec/cbor_any.cr @@ -17,14 +17,14 @@ describe CBOR do end it "from array" do - array = CBOR::Any.from_cbor [ "a", "b", "c", "d" ].to_cbor + array = CBOR::Any.from_cbor ["a", "b", "c", "d"].to_cbor array.to_cbor.hexstring.should eq "846161616261636164" end it "from hash" do h_cany = Hash(String | Int32, String | Int32).new h_cany["name"] = "Alice" - h_cany["age"] = 30 + h_cany["age"] = 30 h_cany["size"] = 160 cbor_any_hash = CBOR::Any.from_cbor h_cany.to_cbor cbor_any_hash.to_cbor.hexstring.should eq "a3646e616d6565416c69636563616765181e6473697a6518a0" diff --git a/src/cbor/any.cr b/src/cbor/any.cr index 36c2db9..1d6aa65 100644 --- a/src/cbor/any.cr +++ b/src/cbor/any.cr @@ -22,7 +22,6 @@ # when the underlying value is not a String will raise: the value won't automatically # be converted (parsed) to a `String`. struct CBOR::Any - # All possible CBOR types. alias Type = Nil | Bool | String | Bytes | Int8 | UInt8 | Int16 | UInt16 | Int32 | UInt32 | Int64 | UInt64 | Int128 | @@ -50,42 +49,36 @@ struct CBOR::Any new value when Time new value - when Array(Type) ary = [] of CBOR::Any value.each do |v| ary << new(v) end new ary - when Array(JSON::Any) ary = [] of CBOR::Any value.each do |v| ary << new(v) end new ary - when Array(CBOR::Type) ary = [] of CBOR::Any value.each do |v| ary << new(v) end new ary - when Hash(String, JSON::Any) hash = {} of String => CBOR::Any value.each do |key, v| hash[key] = new(v) end new hash - when Hash(CBOR::Type, CBOR::Type) hash = {} of CBOR::Any => CBOR::Any value.each do |key, v| hash[new(key)] = new(v) end new hash - when Hash(String, Type) hash = {} of String => CBOR::Any value.each do |key, v| diff --git a/src/cbor/encoder.cr b/src/cbor/encoder.cr index 67bf1c4..a727b85 100644 --- a/src/cbor/encoder.cr +++ b/src/cbor/encoder.cr @@ -49,7 +49,6 @@ class CBOR::Encoder write(value.to_rfc3339) end - def write(value : Float32 | Float64) case value when Float32 diff --git a/src/cbor/from_cbor.cr b/src/cbor/from_cbor.cr index 08f1be3..2b20677 100644 --- a/src/cbor/from_cbor.cr +++ b/src/cbor/from_cbor.cr @@ -136,23 +136,23 @@ end def Time.new(decoder : CBOR::Decoder) : Time # In case Time is formatted as a JSON#to_json String. value = case decoder.current_token - when CBOR::Token::StringT - Time::Format::RFC_3339.parse(decoder.read_string) - else - case tag = decoder.read_tag - when CBOR::Tag::RFC3339Time - Time::Format::RFC_3339.parse(decoder.read_string) - when CBOR::Tag::EpochTime - case num = decoder.read_num - when Int - Time.unix(num) - when Float - Time.unix_ms((BigFloat.new(num) * 1_000).to_u64) - end - else - raise CBOR::ParseError.new("Expected tag to have value 0 or 1, got #{tag.value}") - end - end + when CBOR::Token::StringT + Time::Format::RFC_3339.parse(decoder.read_string) + else + case tag = decoder.read_tag + when CBOR::Tag::RFC3339Time + Time::Format::RFC_3339.parse(decoder.read_string) + when CBOR::Tag::EpochTime + case num = decoder.read_num + when Int + Time.unix(num) + when Float + Time.unix_ms((BigFloat.new(num) * 1_000).to_u64) + end + else + raise CBOR::ParseError.new("Expected tag to have value 0 or 1, got #{tag.value}") + end + end unless value raise CBOR::ParseError.new("could not parse time representation") -- 2.30.2
builds.sr.ht <builds@sr.ht>crystal-cbor/patches/.build.yml: SUCCESS in 28s [Rewrite of CBOR::Any patchset (with better source format)][0] v2 from [~karchnu][1] [0]: https://lists.sr.ht/~arestifo/crystal-cbor/patches/23445 [1]: mailto:karchnu@karchnu.fr ✓ #531503 SUCCESS crystal-cbor/patches/.build.yml https://builds.sr.ht/~arestifo/job/531503