iddlmZddlmZddlmZddlmZddlZddl m Z m Z m Z m Z mZmZmZej"dk\rddl mZGd d Zndd lmZmZdd lmZdd lmZmZmZddlmZddlmZmZm Z m!Z!m"Z"ddl#m$Z$ddl%m&Z'ddl(m)Z)m*Z*m+Z+m,Z,ddl-m.Z.m/Z/ddl0Z0e r*ddl1Z2ddl3m4Z4ddl%m5Z6ddl7m8Z8m9Z9ddl:m;Z;ddlZ>ddl?m@Z@mAZAmBZBGddeZCGddeCZDGd d!eEZFy)") annotations)abstractmethod) timedelta)PathN) TYPE_CHECKINGDictIterableListLiteralOptionalUnion) )overridec eZdZy)EnforceOverridesN)__name__ __module__ __qualname__F/mnt/e/genesis-system/.venv/lib/python3.12/site-packages/lancedb/db.pyrrs rr)rr)EmbeddingFunctionRegistry)data_to_reader sanitize_urivalidate_schema)LOOP)ListNamespacesResponseCreateNamespaceResponseDropNamespaceResponseDescribeNamespaceResponseListTablesResponse) __version__)connect) AsyncTable LanceTableTablesanitize_create_table)get_uri_schemevalidate_table_name) LanceModel) Connection)DATAURI)EmbeddingFunctionConfig)StorageOptionsProvider)Session) _normalize_create_namespace_mode_normalize_drop_namespace_mode"_normalize_drop_namespace_behaviorceZdZdZ d ddZ d ddZ d ddZddZ d ddZe ddd dd Z e ddddddd  dd Z dd Z ddddd  d dZ d!d"dZ d d#dZdZd!d$dZed%dZy)& DBConnectionz'An active LanceDB connection interface.Nc$|g}tgdS)aList immediate child namespace names in the given namespace. Parameters ---------- namespace: List[str], default [] The parent namespace to list namespaces in. Empty list represents root namespace. page_token: str, optional Token for pagination. Use the token from a previous response to get the next page of results. limit: int, optional The maximum number of results to return. Returns ------- ListNamespacesResponse Response containing namespace names and optional page_token for pagination. N) namespaces page_token)rself namespacer:limits rlist_namespaceszDBConnection.list_namespacesDs0  I%EErctd)-Create a new namespace. Parameters ---------- namespace: List[str] The namespace identifier to create. mode: str, optional Creation mode - "create" (fail if exists), "exist_ok" (skip if exists), or "overwrite" (replace if exists). Case insensitive. properties: Dict[str, str], optional Properties to set on the namespace. Returns ------- CreateNamespaceResponse Response containing the properties of the created namespace. ?Namespace operations are not supported for this connection typeNotImplementedErrorr<r=mode propertiess rcreate_namespacezDBConnection.create_namespace`." M  rctd) Drop a namespace. Parameters ---------- namespace: List[str] The namespace identifier to drop. mode: str, optional Whether to skip if not exists ("SKIP") or fail ("FAIL"). Case insensitive. behavior: str, optional Whether to restrict drop if not empty ("RESTRICT") or cascade ("CASCADE"). Case insensitive. Returns ------- DropNamespaceResponse Response containing properties and transaction_id if applicable. rBrCr<r=rFbehaviors rdrop_namespacezDBConnection.drop_namespace{rIrctd)Describe a namespace. Parameters ---------- namespace: List[str] The namespace identifier to describe. Returns ------- DescribeNamespaceResponse Response containing the namespace properties. rBrCr<r=s rdescribe_namespacezDBConnection.describe_namespaces" M  rctd)qList all tables in this database with pagination support. Parameters ---------- namespace: List[str], optional The namespace to list tables in. None or empty list represents root namespace. page_token: str, optional Token for pagination. Use the token from a previous response to get the next page of results. limit: int, optional The maximum number of results to return. Returns ------- ListTablesResponse Response containing table names and optional page_token for pagination. z5list_tables is not supported for this connection typerCr;s r list_tableszDBConnection.list_tabless0" C  rr=cy)a3List all tables in this database, in sorted order Parameters ---------- namespace: List[str], default [] The namespace to list tables in. Empty list represents root namespace. page_token: str, optional The token to use for pagination. If not present, start from the beginning. Typically, this token is last table name from the previous page. limit: int, default 10 The size of the page to return. Returns ------- Iterable of str Nr)r<r:r>r=s r table_nameszDBConnection.table_namess2 rr=storage_optionsstorage_options_providerdata_storage_versionenable_v2_manifest_pathsc t)aHCreate a [Table][lancedb.table.Table] in the database. Parameters ---------- name: str The name of the table. namespace: List[str], default [] The namespace to create the table in. Empty list represents root namespace. data: The data to initialize the table, *optional* User must provide at least one of `data` or `schema`. Acceptable types are: - list-of-dict - pandas.DataFrame - pyarrow.Table or pyarrow.RecordBatch schema: The schema of the table, *optional* Acceptable types are: - pyarrow.Schema - [LanceModel][lancedb.pydantic.LanceModel] mode: str; default "create" The mode to use when creating the table. Can be either "create" or "overwrite". By default, if the table already exists, an exception is raised. If you want to overwrite the table, use mode="overwrite". exist_ok: bool, default False If a table by the same name already exists, then raise an exception if exist_ok=False. If exist_ok=True, then open the existing table; it will not add the provided data but will validate against any schema that's specified. on_bad_vectors: str, default "error" What to do if any of the vectors are not the same size or contains NaNs. One of "error", "drop", "fill". fill_value: float The value to use when filling vectors. Only used if on_bad_vectors="fill". storage_options: dict, optional Additional options for the storage backend. Options already set on the connection will be inherited by the table, but can be overridden here. See available options at To enable stable row IDs (row IDs remain stable after compaction, update, delete, and merges), set `new_table_enable_stable_row_ids` to `"true"` in storage_options when connecting to the database. data_storage_version: optional, str, default "stable" Deprecated. Set `storage_options` when connecting to the database and set `new_table_data_storage_version` in the options. enable_v2_manifest_paths: optional, bool, default False Deprecated. Set `storage_options` when connecting to the database and set `new_table_enable_v2_manifest_paths` in the options. Returns ------- LanceTable A reference to the newly created table. !!! note The vector index won't be created by default. To create the index, call the `create_index` method on the table. Examples -------- Can create with list of tuples or dictionaries: >>> import lancedb >>> db = lancedb.connect("./.lancedb") >>> data = [{"vector": [1.1, 1.2], "lat": 45.5, "long": -122.7}, ... {"vector": [0.2, 1.8], "lat": 40.1, "long": -74.1}] >>> db.create_table("my_table", data) LanceTable(name='my_table', version=1, ...) >>> db["my_table"].head() pyarrow.Table vector: fixed_size_list[2] child 0, item: float lat: double long: double ---- vector: [[[1.1,1.2],[0.2,1.8]]] lat: [[45.5,40.1]] long: [[-122.7,-74.1]] You can also pass a pandas DataFrame: >>> import pandas as pd >>> data = pd.DataFrame({ ... "vector": [[1.1, 1.2], [0.2, 1.8]], ... "lat": [45.5, 40.1], ... "long": [-122.7, -74.1] ... }) >>> db.create_table("table2", data) LanceTable(name='table2', version=1, ...) >>> db["table2"].head() pyarrow.Table vector: fixed_size_list[2] child 0, item: float lat: double long: double ---- vector: [[[1.1,1.2],[0.2,1.8]]] lat: [[45.5,40.1]] long: [[-122.7,-74.1]] Data is converted to Arrow before being written to disk. For maximum control over how data is saved, either provide the PyArrow schema to convert to or else provide a [PyArrow Table](pyarrow.Table) directly. >>> import pyarrow as pa >>> custom_schema = pa.schema([ ... pa.field("vector", pa.list_(pa.float32(), 2)), ... pa.field("lat", pa.float32()), ... pa.field("long", pa.float32()) ... ]) >>> db.create_table("table3", data, schema = custom_schema) LanceTable(name='table3', version=1, ...) >>> db["table3"].head() pyarrow.Table vector: fixed_size_list[2] child 0, item: float lat: float long: float ---- vector: [[[1.1,1.2],[0.2,1.8]]] lat: [[45.5,40.1]] long: [[-122.7,-74.1]] It is also possible to create an table from `[Iterable[pa.RecordBatch]]`: >>> import pyarrow as pa >>> def make_batches(): ... for i in range(5): ... yield pa.RecordBatch.from_arrays( ... [ ... pa.array([[3.1, 4.1], [5.9, 26.5]], ... pa.list_(pa.float32(), 2)), ... pa.array(["foo", "bar"]), ... pa.array([10.0, 20.0]), ... ], ... ["vector", "item", "price"], ... ) >>> schema=pa.schema([ ... pa.field("vector", pa.list_(pa.float32(), 2)), ... pa.field("item", pa.utf8()), ... pa.field("price", pa.float32()), ... ]) >>> db.create_table("table4", make_batches(), schema=schema) LanceTable(name='table4', version=1, ...) rC)r<namedataschemarFexist_okon_bad_vectors fill_valueembedding_functionsr=rZr[r\r]s r create_tablezDBConnection.create_tables Z"!rc$|j|SN) open_tabler<r_s r __getitem__zDBConnection.__getitem__st$$rr=rZr[index_cache_sizect)aDOpen a Lance Table in the database. Parameters ---------- name: str The name of the table. namespace: List[str], optional The namespace to open the table from. None or empty list represents root namespace. index_cache_size: int, default 256 **Deprecated**: Use session-level cache configuration instead. Create a Session with custom cache sizes and pass it to lancedb.connect(). Set the size of the index cache, specified as a number of entries The exact meaning of an "entry" will depend on the type of index: * IVF - there is one entry for each IVF partition * BTREE - there is one entry for the entire index This cache applies to the entire opened table, across all indices. Setting this value higher will increase performance on larger datasets at the expense of more RAM storage_options: dict, optional Additional options for the storage backend. Options already set on the connection will be inherited by the table, but can be overridden here. See available options at Returns ------- A LanceTable object representing the table. rC)r<r_r=rZr[rms rrizDBConnection.open_tables R"!rc|g}t)aDrop a table from the database. Parameters ---------- name: str The name of the table. namespace: List[str], default [] The namespace to drop the table from. Empty list represents root namespace. rC)r<r_r=s r drop_tablezDBConnection.drop_tables  I!!rc|g}|g}t)Rename a table in the database. Parameters ---------- cur_name: str The current name of the table. new_name: str The new name of the table. cur_namespace: List[str], optional The namespace of the current table. None or empty list represents root namespace. new_namespace: List[str], optional The namespace to move the table to. If not specified, defaults to the same as cur_namespace. rCr<cur_namenew_name cur_namespace new_namespaces r rename_tablezDBConnection.rename_tables ,  M  M!!rct)Y Drop database This is the same thing as dropping all the tables rCr<s r drop_databasezDBConnection.drop_databases "!rc|g}t)z Drop all tables from the database Parameters ---------- namespace: List[str], optional The namespace to drop all tables from. None or empty list represents root namespace. rCrQs rdrop_all_tableszDBConnection.drop_all_tabless  I!!rc|jSrh)_urir{s rurizDBConnection.uris yyrNNNr=Optional[List[str]]r: Optional[str]r> Optional[int]returnrNNr= List[str]rFrrGOptional[Dict[str, str]]rrr=rrFrrMrrr r=rrr!r=rr:rr>rrr"N r:rr>intr=rr Iterable[str]NNcreateFerrorN)r_strr`Optional[DATA]ra&Optional[Union[pa.Schema, LanceModel]]rFrrbboolrcrrdfloatre'Optional[List[EmbeddingFunctionConfig]]r=rrZrr["Optional['StorageOptionsProvider']r\rr]Optional[bool]rr()r_rrr') r_rr=rrZrr[rrmrrr(rh)r_rr=rrtrrurrvrrwrr=rrr)rrr__doc__r?rHrNrRrUrrXrfrkrirprxr|r~propertyrrrrr7r7As1*.$(# F&F"F F F>#/3    -  !  <#"&        6 &*.$(#  & "     8%) *.  !  '     4 $9=%GKl"*.48GK.237l"l"l"7 l"  l"  l"l"l"El"'l"2l"#El",l"#1l" !l"l"\%*.48GK*.)")"' )" 2 )" #E )"()" )"V "&.2-1 """+ " + "8" "rr7ceZdZdZddddd d$dZed%dZed&dZed'dZe d(dZ d'd Z d)d Z ed*d Z e d+ d,d Ze d- d.d Ze d- d/dZed0dZe d+ d1dZe d2dd d3dZd4dZd5dZe d6dddddd d7dZeddddd d8dZddddd d9dZe d: d;dZeddZej<dd ed!"ed#Z y)?LanceDBConnectiona A connection to a LanceDB database. Parameters ---------- uri: str or Path The root uri of the database. read_consistency_interval: timedelta, default None The interval at which to check for updates to the table from other processes. If None, then consistency is not checked. For performance reasons, this is the default. For strong consistency, set this to zero seconds. Then every read will check for updates from other processes. As a compromise, you can set this to a non-zero timedelta for eventual consistency. If more than that interval has passed since the last check, then the table will be checked for updates. Note: this consistency only applies to read operations. Write operations are always consistent. Examples -------- >>> import lancedb >>> db = lancedb.connect("./.lancedb") >>> db.create_table("my_table", data=[{"vector": [1.1, 1.2], "b": 2}, ... {"vector": [0.5, 1.3], "b": 4}]) LanceTable(name='my_table', version=1, ...) >>> db.create_table("another_table", data=[{"vector": [0.4, 0.4], "b": 6}]) LanceTable(name='another_table', version=1, ...) >>> sorted(db.table_names()) ['another_table', 'my_table'] >>> len(db) 2 >>> db["my_table"] LanceTable(name='my_table', version=1, ...) >>> "my_table" in db True >>> db.drop_table("my_table") >>> db.drop_table("another_table") N)read_consistency_intervalrZsession_innerc |||_ytts t}ttxsdk(}|rttrnj drddnj drddt jdk(r#tdk\rdd k(r d d k(rd dtjjtjd d ||j nd fd}|_ ttj ||_y)Nfilezfile://zfile:/win32rr/:r#T)parentsrbc VKttddddd{S7wrh)lancedb_connectr)read_consistency_interval_secsrrZrsr do_connectz.LanceDBConnection.__init__..do_connectOs;(S!.    s )'))_conn isinstancerr*r startswithsysplatformlen expanduserabsolutemkdir total_secondsrZAsyncConnectionrrun) r<rrrZrrschemeis_localrrs ` `` @r__init__zLanceDBConnection.__init__(s%  DJ #t$#C(Fc4(>),ab'C^^H-ab'C<<7*3x1}Q33q6S=!!"g3i.."++-C IOOD4O 8 $ 0-F-T-T-V *-1 * " /$TXXjl%;< rc\tj|jjSrh)rrrget_read_consistency_intervalr{s rrz+LanceDBConnection.read_consistency_intervalcsxx @@BCCrc.|jjSrh)rrr{s rrzLanceDBConnection.sessiongszz!!!rc.|jjSrh)rrr{s rrzLanceDBConnection.urikszz~~rc|d|S)Nrr)clsinners r from_innerzLanceDBConnection.from_inneros4&&rc|jjd|jj}|j|dt |jz }|dz }|S)Nz(uri=z, read_consistency_interval=)) __class__rrrrrepr)r<vals r__repr__zLanceDBConnection.__repr__ss^(()tzz~~.@A  ) ) 5 1$t7U7U2V1WX XC s  rcKtt|jd{}|j||d{S7#7w)N) start_afterr>)rrrrX)r<rr>conns r_async_get_table_namesz(LanceDBConnection._async_get_table_nameszs>_TXX%>>?%%+U%KKK ?Ks AAAAAAc.|jjSrh)rrr{s rrzLanceDBConnection._inner~szz   rcl|g}tj|jj|||S)aList immediate child namespace names in the given namespace. Parameters ---------- namespace: List[str], optional The parent namespace to list namespaces in. None or empty list represents root namespace. page_token: str, optional Token for pagination. Use the token from a previous response to get the next page of results. limit: int, optional The maximum number of results to return. Returns ------- ListNamespacesResponse Response containing namespace names and optional page_token for pagination. r=r:r>)rrrr?r;s rr?z!LanceDBConnection.list_namespacess>2  Ixx JJ & &# % '   rcdtj|jj|||S)rA)r=rFrG)rrrrHrEs rrHz"LanceDBConnection.create_namespaces30xx JJ ' '#$: (   rcdtj|jj|||S)rK)r=rFrM)rrrrNrLs rrNz LanceDBConnection.drop_namespaces.0xx JJ % % x % X  rc`tj|jj|S)rPrV)rrrrRrQs rrRz$LanceDBConnection.describe_namespaces%xx 55 5JKKrcl|g}tj|jj|||S)rTr)rrrrUr;s rrUzLanceDBConnection.list_tabless>2  Ixx JJ " "# % #   rrVcddl}|jdtd|g}tj|j j |||S)a5Get the names of all tables in the database. The names are sorted. .. deprecated:: Use :meth:`list_tables` instead, which provides proper pagination support. Parameters ---------- namespace: List[str], optional The namespace to list tables in. page_token: str, optional The token to use for pagination. limit: int, default 10 The maximum number of tables to return. Returns ------- Iterator of str. A list of table names. rN6table_names() is deprecated, use list_tables() insteadr stacklevelr=rr>)warningswarnDeprecationWarningrrrrX)r<r:r>r=rs rrXzLanceDBConnection.table_namess]6  D    Ixx JJ " "#5 #   rc4t|jSrh)rrXr{s r__len__zLanceDBConnection.__len__8s4##%&&rc&||jvSrh)rXrjs r __contains__zLanceDBConnection.__contains__;st'')))rrYc | g} |jdvr tdt|tj|||||||||| | |  }|S)zCreate a table in the database. Parameters ---------- namespace: List[str], optional The namespace to create the table in. See --- DBConnection.create_table )r overwritez+mode must be either 'create' or 'overwrite')rFrbrcrdrer=rZr[)lower ValueErrorr+r'r)r<r_r`rarFrbrcrdrer=rZr[r\r]tbls rrfzLanceDBConnection.create_table>sm:  I ::<6 6JK KD!    )! 3+%=   rrlc||g}|ddl}|jdtdtj||||||S)a0Open a table in the database. Parameters ---------- name: str The name of the table. namespace: List[str], optional The namespace to open the table from. Returns ------- A LanceTable object representing the table. Nrzindex_cache_size is deprecated. Use session-level cache configuration instead. Create a Session with custom cache sizes and pass it to lancedb.connect().rrrl)rrrr'open)r<r_r=rZr[rmrs rrizLanceDBConnection.open_tableqs[.  I  '  MM4#    +%=-   rTtarget_namespacesource_version source_tag is_shallowc |g}tj|jj||||||t j |||S)arClone a table from a source table. A shallow clone creates a new table that shares the underlying data files with the source table but has its own independent manifest. This allows both the source and cloned tables to evolve independently while initially sharing the same data, deletion, and index files. Parameters ---------- target_table_name: str The name of the target table to create. source_uri: str The URI of the source table to clone from. target_namespace: List[str], optional The namespace for the target table. None or empty list represents root namespace. source_version: int, optional The version of the source table to clone. source_tag: str, optional The tag of the source table to clone. is_shallow: bool, default True Whether to perform a shallow clone (True) or deep clone (False). Currently only shallow clone is supported. Returns ------- A LanceTable object representing the cloned table. rrV)rrr clone_tabler'r)r<target_table_name source_urirrrrs rrzLanceDBConnection.clone_tablesbL  #!   JJ " "!!1-%% #    &  rcn|g}tj|jj|||y)aBDrop a table from the database. Parameters ---------- name: str The name of the table. namespace: List[str], optional The namespace to drop the table from. ignore_missing: bool, default False If True, ignore if the table does not exist. Nr=ignore_missing)rrrrp)r<r_r=rs rrpzLanceDBConnection.drop_tables9$  I  JJ ! ! . "  rcj|g}tj|jj|y)NrVrrrr~rQs rr~z!LanceDBConnection.drop_all_tabless+  I ++i+@Arcx|g}|g}tj|jj||||y)aRename a table in the database. Parameters ---------- cur_name: str The current name of the table. new_name: str The new name of the table. cur_namespace: List[str], optional The namespace of the current table. new_namespace: List[str], optional The namespace to move the table to. Nrvrw)rrrrxrss rrxzLanceDBConnection.rename_tablesI*  M  M  JJ # #++ $  r0.15.10.17Use drop_all_tables() instead deprecated_in removed_incurrent_versiondetailsc^tj|jjyrhrr{s rr|zLanceDBConnection.drop_databases ++-.r) rr/rOptional[timedelta]rZrrOptional[Session]rzOptional[LanceDbConnection]rr )rr r)rLanceDbConnection)rrr>r)rrrrrrrrrrr)rr)r_rrrr)r_rr`rrarrFrrbrrcrrdrrerr=rrZrr[rr\rr]rrr') r_rr=rrZrr[rrmrrr')rrrrrrrrrrrrrr')NFr_rr=rrrrhrr)!rrrrrrrrr classmethodrrrrrr?rHrNrRrUrXrrrfrirrpr~rx deprecation deprecatedr$r|rrrrrs%V:>48%).29= 9=$7 9= 2 9= # 9=,9=vDD""''L!!*.$(#  & "     @#/3    -  !   :#"&         6 L L*.$(#  & "      @%)' *. ' !' ' ' '  ' ' R'* $9=%GK0*.48GK.2370007 0  0  000E0'020#E0,0#10 !00d *.48GK*.* * ' * 2 * #E * (*  * * b15(,$(6 6 6 . 6 & 6 "6 6  6 p*.$   '    2BB  .2-1    +  +   B[#/  / /rrc eZdZdZd#dZdZdZdZdZdZ e d$dZ d%d Z d& d'd Z d( d)d Z d( d*d Z d+dZ d& d,dZd d d d d-dZ d.d d d d d/dZd d d d d d d0dZd d d dd d1dZ d( d2dZd dd d3dZd4d5dZej4dded !d"Zy )6ra1An active LanceDB connection To obtain a connection you can use the [connect_async][lancedb.connect_async] function. This could be a native connection (using lance) or a remote connection (e.g. for connecting to LanceDb Cloud) Local connections do not currently hold any open resources but they may do so in the future (for example, for shared cache or connections to catalog services) Remote connections represent an open connection to the remote server. The [close][lancedb.db.AsyncConnection.close] method can be used to release any underlying resources eagerly. The connection can also be used as a context manager. Connections can be shared on multiple threads and are expected to be long lived. Connections can also be used as a context manager, however, in many cases a single connection can be used for the lifetime of the application and so this is often not needed. Closing a connection is optional. If it is not closed then it will be automatically closed when the connection object is deleted. Examples -------- >>> import lancedb >>> async def doctest_example(): ... with await lancedb.connect_async("/tmp/my_dataset") as conn: ... # do something with the connection ... pass ... # conn is closed here c||_yrhr)r< connections rrzAsyncConnection.__init__Cs   rc6|jjSrh)rrr{s rrzAsyncConnection.__repr__Fs{{##%%rc|Srhrr{s r __enter__zAsyncConnection.__enter__Is rc$|jyrh)close)r<_s r__exit__zAsyncConnection.__exit__Ls  rc6|jjS)z&Return True if the connection is open.)ris_openr{s rrzAsyncConnection.is_openOs{{""$$rc8|jjy)zClose the connection, releasing any underlying resources. It is safe to call this method multiple times. Any attempt to use the connection after it is closed will result in an error.N)rrr{s rrzAsyncConnection.closeSs rc.|jjSrh)rrr{s rrzAsyncConnection.uri[s{{rcpK|jjd{}| t|Sy7w)N)seconds)rrr)r< interval_secss rrz-AsyncConnection.get_read_consistency_interval_s5"kkGGII  $]3 3 Js 646NcxK|g}|jj|||d{}tdi|S7w)a^List immediate child namespace names in the given namespace. Parameters ---------- namespace: List[str], optional The parent namespace to list namespaces in. None or empty list represents root namespace. page_token: str, optional The token to use for pagination. If not present, start from the beginning. limit: int, optional The maximum number of results to return. Returns ------- ListNamespacesResponse Response containing namespace names and optional pagination token Nrr)rr?rr<r=r:r>results rr?zAsyncConnection.list_namespacesfsM.  I{{22Je3  &/// &:8:cK|jj|t||d{}tdi|S7w)aCreate a new namespace. Parameters ---------- namespace: List[str] The namespace identifier to create. mode: str, optional Creation mode - "create", "exist_ok", or "overwrite". Case insensitive. properties: Dict[str, str], optional Properties to associate with the namespace Returns ------- CreateNamespaceResponse Response containing namespace properties )rFrGNr)rrHr3r)r<r=rFrGr's rrHz AsyncConnection.create_namespacesI,{{33 1$7!4   '000  s +?=?cK|jj|t|t|d{}t di|S7w)rK)rFrMNr)rrNr4r5r )r<r=rFrMr's rrNzAsyncConnection.drop_namespacesN.{{11 /57A2   %.v..  s4AAAcjK|jj|d{}tdi|S7w)rPNr)rrRr!)r<r=r's rrRz"AsyncConnection.describe_namespaces3{{55i@@(2622As 313cxK|g}|jj|||d{}tdi|S7w)rTNrr)rrUr"r&s rrUzAsyncConnection.list_tablessM0  I{{..Je/  "+F++ r(rcKddl}|jdtd|g}|jj |||d{S7w)aList all tables in this database, in sorted order .. deprecated:: Use :meth:`list_tables` instead, which provides proper pagination support. Parameters ---------- namespace: List[str], optional The namespace to list tables in. None or empty list represents root namespace. start_after: str, optional If present, only return names that come lexicographically after the supplied value. This can be combined with limit to implement pagination by setting this to the last table name from the previous page. limit: int, default 10 The number of results to return. Returns ------- Iterable of str rNrrrr)rrrrrX)r<r=rr>rs rrXzAsyncConnection.table_namess_<  D    I[[,,[-    sAA A A )r=relocationc  K| g} d} | %tj}|j| } |d}|d}t||| ||\}}t ||d}|d}|dk(r|rd}|5|j j |||| || | d{}t|St||}|j j|||| || | d{}t|S7P7w)aCreate an [AsyncTable][lancedb.table.AsyncTable] in the database. Parameters ---------- name: str The name of the table. namespace: List[str], default [] The namespace to create the table in. Empty list represents root namespace. data: The data to initialize the table, *optional* User must provide at least one of `data` or `schema`. Acceptable types are: - list-of-dict - pandas.DataFrame - pyarrow.Table or pyarrow.RecordBatch schema: The schema of the table, *optional* Acceptable types are: - pyarrow.Schema - [LanceModel][lancedb.pydantic.LanceModel] mode: Literal["create", "overwrite"]; default "create" The mode to use when creating the table. Can be either "create" or "overwrite". By default, if the table already exists, an exception is raised. If you want to overwrite the table, use mode="overwrite". exist_ok: bool, default False If a table by the same name already exists, then raise an exception if exist_ok=False. If exist_ok=True, then open the existing table; it will not add the provided data but will validate against any schema that's specified. on_bad_vectors: str, default "error" What to do if any of the vectors are not the same size or contains NaNs. One of "error", "drop", "fill". fill_value: float The value to use when filling vectors. Only used if on_bad_vectors="fill". storage_options: dict, optional Additional options for the storage backend. Options already set on the connection will be inherited by the table, but can be overridden here. See available options at To enable stable row IDs (row IDs remain stable after compaction, update, delete, and merges), set `new_table_enable_stable_row_ids` to `"true"` in storage_options when connecting to the database. Returns ------- AsyncTable A reference to the newly created table. !!! note The vector index won't be created by default. To create the index, call the `create_index` method on the table. Examples -------- Can create with list of tuples or dictionaries: >>> import lancedb >>> async def doctest_example(): ... db = await lancedb.connect_async("./.lancedb") ... data = [{"vector": [1.1, 1.2], "lat": 45.5, "long": -122.7}, ... {"vector": [0.2, 1.8], "lat": 40.1, "long": -74.1}] ... my_table = await db.create_table("my_table", data) ... print(await my_table.query().limit(5).to_arrow()) >>> import asyncio >>> asyncio.run(doctest_example()) pyarrow.Table vector: fixed_size_list[2] child 0, item: float lat: double long: double ---- vector: [[[1.1,1.2],[0.2,1.8]]] lat: [[45.5,40.1]] long: [[-122.7,-74.1]] You can also pass a pandas DataFrame: >>> import pandas as pd >>> data = pd.DataFrame({ ... "vector": [[1.1, 1.2], [0.2, 1.8]], ... "lat": [45.5, 40.1], ... "long": [-122.7, -74.1] ... }) >>> async def pandas_example(): ... db = await lancedb.connect_async("./.lancedb") ... my_table = await db.create_table("table2", data) ... print(await my_table.query().limit(5).to_arrow()) >>> asyncio.run(pandas_example()) pyarrow.Table vector: fixed_size_list[2] child 0, item: float lat: double long: double ---- vector: [[[1.1,1.2],[0.2,1.8]]] lat: [[45.5,40.1]] long: [[-122.7,-74.1]] Data is converted to Arrow before being written to disk. For maximum control over how data is saved, either provide the PyArrow schema to convert to or else provide a [PyArrow Table](pyarrow.Table) directly. >>> import pyarrow as pa >>> custom_schema = pa.schema([ ... pa.field("vector", pa.list_(pa.float32(), 2)), ... pa.field("lat", pa.float32()), ... pa.field("long", pa.float32()) ... ]) >>> async def with_schema(): ... db = await lancedb.connect_async("./.lancedb") ... my_table = await db.create_table("table3", data, schema = custom_schema) ... print(await my_table.query().limit(5).to_arrow()) >>> asyncio.run(with_schema()) pyarrow.Table vector: fixed_size_list[2] child 0, item: float lat: float long: float ---- vector: [[[1.1,1.2],[0.2,1.8]]] lat: [[45.5,40.1]] long: [[-122.7,-74.1]] It is also possible to create an table from `[Iterable[pa.RecordBatch]]`: >>> import pyarrow as pa >>> def make_batches(): ... for i in range(5): ... yield pa.RecordBatch.from_arrays( ... [ ... pa.array([[3.1, 4.1], [5.9, 26.5]], ... pa.list_(pa.float32(), 2)), ... pa.array(["foo", "bar"]), ... pa.array([10.0, 20.0]), ... ], ... ["vector", "item", "price"], ... ) >>> schema=pa.schema([ ... pa.field("vector", pa.list_(pa.float32(), 2)), ... pa.field("item", pa.utf8()), ... pa.field("price", pa.float32()), ... ]) >>> async def iterable_example(): ... db = await lancedb.connect_async("./.lancedb") ... await db.create_table("table4", make_batches(), schema=schema) >>> asyncio.run(iterable_example()) NrrFrrb)r=rZr[r.) r get_instanceget_table_metadatar)rrcreate_empty_tablerrfr&)r<r_r`rarFrbrcrdrZr[r=rer.metadataregistry new_tables rrfzAsyncConnection.create_tables=Z  I  *1==?H223FGH  !$N  J, &(NJ f   H <D 8 D <"kk<<# /)A!=I*)$$"$/D"kk66# /)A!7I)$$+s%B C" CAC"C C" C"r=rZr[rmr.c~K|g}|jj||||||d{}t|S7w)a(Open a Lance Table in the database. Parameters ---------- name: str The name of the table. namespace: List[str], optional The namespace to open the table from. None or empty list represents root namespace. storage_options: dict, optional Additional options for the storage backend. Options already set on the connection will be inherited by the table, but can be overridden here. See available options at index_cache_size: int, default 256 **Deprecated**: Use session-level cache configuration instead. Create a Session with custom cache sizes and pass it to lancedb.connect(). Set the size of the index cache, specified as a number of entries The exact meaning of an "entry" will depend on the type of index: * IVF - there is one entry for each IVF partition * BTREE - there is one entry for the entire index This cache applies to the entire opened table, across all indices. Setting this value higher will increase performance on larger datasets at the expense of more RAM location: str, optional The explicit location (URI) of the table. If provided, the table will be opened from this location instead of deriving it from the database URI and table name. Returns ------- A LanceTable object representing the table. Nr6)rrir&)r<r_r=rZr[rmr.tables rrizAsyncConnection.open_tablesV\  Ikk,, +%=- -  %   )=;=Trc~K|g}|jj||||||d{}t|S7w)asClone a table from a source table. A shallow clone creates a new table that shares the underlying data files with the source table but has its own independent manifest. This allows both the source and cloned tables to evolve independently while initially sharing the same data, deletion, and index files. Parameters ---------- target_table_name: str The name of the target table to create. source_uri: str The URI of the source table to clone from. target_namespace: List[str], optional The namespace for the target table. None or empty list represents root namespace. source_version: int, optional The version of the source table to clone. source_tag: str, optional The tag of the source table to clone. is_shallow: bool, default True Whether to perform a shallow clone (True) or deep clone (False). Currently only shallow clone is supported. Returns ------- An AsyncTable object representing the cloned table. Nr)rrr&)r<rrrrrrr8s rrzAsyncConnection.clone_table9sWL  #! kk--  -)!! .  %   r9cnK|g}|g}|jj||||d{y7w)rrNr)rrxrss rrxzAsyncConnection.rename_tableksH,  M  Mkk&& hm='   s +535FrcK|g} |jj||d{y7#t$r!}|s|d|dt|vr|Yd}~yd}~wwxYww)avDrop a table from the database. Parameters ---------- name: str The name of the table. namespace: List[str], default [] The namespace to drop the table from. Empty list represents root namespace. ignore_missing: bool, default False If True, ignore if the table does not exist. NrVzTable 'z' was not found)rrprr)r<r_r=res rrpzAsyncConnection.drop_tablesk&  I ++(((C C C !o.c!f<= s6A 0.0A0 AAAAAc`K|g}|jj|d{y7w)zDrop all tables from the database. Parameters ---------- namespace: List[str], optional The namespace to drop all tables from. None or empty list represents root namespace. NrVrr~rQs rr~zAsyncConnection.drop_all_tabless.  Ikk))I)>>>s $.,.rrrrcTK|jjd{y7w)rzNr?r{s rr|zAsyncConnection.drop_databaseskk))+++s (&()rrrrrrrrrrr)r=rrrr>rrr)NNNNNNNN)r_rr`rrarrFz(Optional[Literal['create', 'overwrite']]rbrrcrrdzOptional[float]rZrr[rr=rrerr.rrr&)r_rr=rrZrr[rrmrr.rrr&)rrrrrrrrrrrrrr&rrrhr)rrrrrrrrrrrrrr?rHrNrRrUrXrfrirrxrpr~rrr$r|rrrrr#se>!&%*.$(# 0&0"0 0 0B#/3 111- 1 ! 1@#"& /// /  /<3"3 "3(*.$(# ,&,", ,  ,D*.%)# ) ') # )  )  ) \ $9=9=#'(,&*48GKb%*.GK"&b%b%b%7 b% 7 b% ! b%&b%$b%2b%#Eb%'b%Eb% b% b%P*.48GK*."&8!8!' 8! 2 8! #E 8!(8! 8! 8!~15(,$(0!0!0! . 0! & 0!"0!0! 0!l.2-1    +  +  D*.$ '   : ?[#/  ,  ,rr)G __future__rabcrdatetimerpathlibrrtypingrrr r r r r version_inforr overrideslancedb.embeddings.registryrlancedb.commonrrrlancedb.background_looprlance_namespacerrr r!r"r$_lancedbr%rr8r&r'r(r)utilr*r+rpyarrowpapydanticr,r-rcommonr.r/ embeddingsr0ior1r2namespace_utilsr3r4r5r7robjectrrrrrWs # PPPw  5AHH(0  $9!3*!|#|~ `/ `/F[ ,f[ ,r