data modeling - App engine datastore denormalization: index properties in the main entity or the denormalized entity? -

-

consider classic example of blog data modelling, have blog entity many properties, , want list latest blogs in page.

it makes sense denormalize blogpost entity blogpostsummary entity shown in list view, avoiding fetching , deserializing many unwanted properties.

class blogpost(db.model):   title = db.stringproperty()   content = db.textproperty()   created = db.dateproperty()   ...  class blogpostsummary(db.model):   title = db.stringproperty()   content_excerpt = db.textproperty()

the question is: entity should hold indexed properties? there 3 options:

1. index properties in both

  • pros:
    • easy query on both entities.
  • cons:
    • maintaining denormalized indexes expensive.

2. index properties in main entity only

  • pros:
    • indexing properties in main entity more safe, denormalized entity treated redundancy.
  • cons:
    • querying list view need double roundtrip datastore: 1 key-only query blogpost entities, followed batch blogpostsummary.

3. index in denormalized entity only

  • pros:
    • the list view can built single query.
  • cons:
    • the main entity cannot queried properties anymore.
    • the indexes occupy more space when denormalized entity child of main entity.

which option work better? there other options?

would double round trip datastore in option 2 problem?

this abstract question not have "correct" answer. choice of data model depends on specific requirements of project, including:

  • usage patterns (how need access different data)
  • update patterns (e.g. separating updated properties stable properties in order reduce write costs)
  • average performance , extreme-case performance requirements (e.g average blog may have 10 posts, popular blog may have 10,000 posts)
  • ability use memcache reduce datastore trips , improve performance
  • data complexity (i.e. how many different kids of entities depend on particular entity kind)
  • transaction requirements
  • security , access roles considerations (e.g. not exposing private data mistake)

by way, there way model data in datastore - using child entities. example, blog posts may child entities of blog entity. way can retrieve blog posts single query providing parent key - without storing post ids or keys in blog entity or blog id/key in post entities.


Comments

Post a Comment

Popular posts from this blog

PHP DOM loadHTML() method unusual warning -

-
i had following code works fine on localhost: $document = new domdocument(); $document->loadhtml($p_result); $form = $document->getelementsbytagname('form')->item(0); // code continues using $form variable after same code updated on outside server, loadhtml() failed , issued warning. warning: domdocument::loadhtml() expects parameter 1 valid path, string given in path/name/to/script.php returned null instead of object code pretty gets fatal error. note contents of $p_result absolutely same on outside server , on localhost. but why displays kind of warning , why not work? doesn't loadhtml() expects argument 1 string in first place? why method expects parameter 1 valid path ? just make clear i'm not calling loadhtmlfile() , i'm calling loadhtml() . thanks. you're affected one of php bugs . issue present in php 5.6.8 , 5.6.9. have affected php version on server, , bug-free version on localhost. the bug forbids null
Read more

python - How to create jsonb index using GIN on SQLAlchemy? -

-
here's current code creating index jsonb. index("mytable_data_idx_id_key", mytable.data['id'].astext, postgresql_using='gin') but got error. sqlalchemy.exc.programmingerror: (psycopg2.programmingerror) data type text has no default operator class access method "gin" hint: must specify operator class index or define default operator class data type. [sql: "create index event_data_idx_id_key on event using gin ((data ->> 'id'))"] is there way create index on sqlalchemy? the postgresql specific sqlalchemy docs @ http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#operator-classes mention postgresql_ops dictionary provide "operator class" used postgresql, , provide example illustrating use: index('my_index', my_table.c.id, my_table.c.data, postgresql_ops={ 'data': 'text_pattern_ops',
Read more

c# - TransactionScope not rolling back although no complete() is called -

-
i'm using transactionscope rollback transaction fail bool errorreported = false; action<importerrorlog> newerrorcallback = e => { errorreported = true; errorcallback(e); }; using (var transaction = new transactionscope()) { foreach (importtaskdefinition task in taskdefinition) { loader.load(streamfile, newerrorcallback, task.destinationtable, processingtaskid); } if (!errorreported) transaction.complete(); } i'm sure there no transactionscope started ahead or after code. i'm using entity framework insert in db. regardless state of errorreported transaction never rolled in case of error. what missing ? transactionscope sets transaction.current . that's does. wants transacted must @ property. i believe ef each time connection opened reason. probably, connection open when scope installed. open connection inside of scope or enlist manually. ef has nasty "design decision": default opens
Read more