Typesense derive enhancements

[haydenhoang]

Change Summary

This PR added support for more attributes:

Collection-level attributes

Added support for token_separators and symbols_to_index:

#[derive(Typesense, Serialize, Deserialize)]
#[typesense(
    collection_name = "kitchen_sink_products",
    token_separators = ["-", "/"],
    symbols_to_index = ["+"]
)]
struct KitchenSinkProduct {}

Field-level attributes

• Boolean attributes: sort, index, store, infix, stem, range_index, optional
• Value-only attributes: locale, vec_dist, num_dim
• Special attributes:
  • type: override the auto inferred type
  • rename: rename a Typesense field (This must be used with #[serde(rename = "new_name")])
  • flatten: generate schema for fields of nested structs
  • skip: skip generating schema for a field

Boolean attributes can be specified as a shorthand flag e.g. #[typesense(index)]or explicitly set a value e.g.#[typesense(index = false)])

Full example:

#[derive(Typesense, Serialize, Deserialize)]
#[typesense(
    collection_name = "mega_products",
    default_sorting_field = "price",
    enable_nested_fields = true,
    token_separators = ["-", "/"],
    symbols_to_index = ["+"]
)]
struct MegaProduct {
    id: String,

    #[typesense(infix, stem)]
    title: String,

    #[typesense(rename = "product_name")]
    #[serde(rename = "product_name")]
    official_name: String,

    #[typesense(facet)]
    brand: String,

    #[typesense(sort)]
    price: f32,

    #[typesense(range_index)]
    review_score: f32,

    #[typesense(index = false, store = false)]
    internal_sku: Option<String>,

    #[typesense(type = "geopoint")]
    location: (f32, f32),

    #[typesense(num_dim = 4, vec_dist = "cosine")]
    embedding: Vec<f32>,

    #[typesense(flatten, skip)]
    details: ProductDetails,

    #[typesense(flatten, rename = "logistics_data")]
    #[serde(rename = "logistics_data")]
    logistics: Logistics,

    manufacturer: Manufacturer,

    #[typesense(flatten)]
    parts: Vec<Part>,

    tags: Option<Vec<String>>,

    #[typesense(rename = "primary_address.city")]
    #[serde(rename = "primary_address.city")]
    primary_city: String,
}

Bug fix

This PR also fixed the partial struct derive wrapping an already-optional field in an Option<>

• I have read and signed the Contributor License Agreement.

[RoDmitry]

the partial struct derive wrapping an already-optional field in an Option<>

I thought it's supposed to wrap. Because there are optional fields possible, no? How else would you update a field and set it to null? That needs to be added to tests.

[RoDmitry]

Don't do it in a runtime, at least use const logic.

[haydenhoang]

I'm not sure how to optimize this. Could you clarify "const logic"?

[haydenhoang]

Would really appreciate if you can optimize it.

[RoDmitry]

Ok, I will look into it later.

[RoDmitry]

Same issue: must not be .flatten().collect() in runtime.

[haydenhoang]

Used .extend() instead of .flatten().collect(), is this the ideal approach?

let mut fields = Vec::new();
#(fields.extend(#typesense_fields);)*

[RoDmitry]

Attributes locale and optional are missing in tests (derive_integration_test.rs)

[haydenhoang]

How else would you update a field and set it to null? That needs to be added to tests.

Yes, you're right. I have reverted the change and added test for it.