a ¯ïaŽ‚ã@sšdZddlZddlZddlZddlZddlmZddlZddlZ ddl Z ddl m Z mZddlmZmZmZmZmZmZmZmZmZddlmZddlmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-dd l.m/Z/d d „Z0d d „Z1dd„Z2dd„Z3Gdd„deƒZ4dd„Z5Gdd„dƒZ6dd„Z7ee ddZ8ee ddZ9ee ddZ:ee ddZ;Gdd „d ƒZ| t|ƒ¡q| |¡q|S)zyConstruct lists from cell arrays (loaded as numpy ndarrays), recursing into items if they contain mat_struct objects.)r#r!ÚappendÚ_matstruct_to_dictr*Ú_inspect_cell_array)r%Z elem_listZsub_elemr(r(r)r-ps  r-cCsTi}|jD]D}|j|}t|tƒr0t|ƒ||<q t|ƒrFt|ƒ||<q |||<q |S)z/Construct nested dicts from mat_struct objects.)Ú _fieldnamesÚ__dict__r#r!r,r*r-)ZmatobjÚdÚfr'r(r(r)r,~s    r,cCsJ|D]@}t||tƒr(t||ƒ||<qt||ƒrt||ƒ||<q|S)z,Convert mat objects in dict to nested dicts.)r#r!r,r*r-)r0Úkeyr(r(r)Ú_simplify_cellsŒs  r3c sbeZdZdZed‡fdd„ ƒZdd„Zd d „Zd d „Zd d„Z ddd„Z ddd„Z dd„Z ‡Z S)ÚMatFile5ReaderaØ Reader for Mat 5 mat files Adds the following attribute to base class uint16_codec - char codec to use for uint16 char arrays (defaults to system default codec) Uses variable reader that has the following stardard interface (see abstract class in ``miobase``:: __init__(self, file_reader) read_header(self) array_from_header(self) and added interface:: set_stream(self, stream) read_full_tag(self) NFTc s>tƒ ||||||||| ¡ | s(t ¡} | |_d|_d|_dS)züInitializer for matlab 5 file format reader %(matstream_arg)s %(load_args)s %(struct_arg)s uint16_codec : {None, string} Set codec to use for uint16 char arrays (e.g., 'utf-8'). Use system default codec if None N)ÚsuperÚ__init__ÚsysÚgetdefaultencodingÚ uint16_codecÚ _file_readerÚ_matrix_reader) ÚselfÚ mat_streamÚ byte_orderZ mat_dtypeZ squeeze_meZchars_as_stringsZmatlab_compatibleZstruct_as_recordÚ verify_compressed_data_integrityr9Úsimplify_cells©Ú __class__r(r)r6ªs ÷ zMatFile5Reader.__init__cCs4|j d¡|j d¡}|j d¡|dkr0dp2dS)z4 Guess byte order. Sets stream pointer to 0 é~érsIMú<ú>)r=ÚseekÚread)r<Úmir(r(r)Úguess_byte_orderÑs   zMatFile5Reader.guess_byte_ordercCsdi}t|jdd}t|j|ƒ}|d ¡ d¡|d<|dd?}|dd@}d ||f|d <|S) z Read in mat 5 file header ÚdtypesÚ file_headerÚ descriptions Z __header__Úversionééÿz%d.%dÚ __version__)rr>r r=ÚitemÚstrip)r<ZhdictZ hdr_dtypeÚhdrZv_majorZv_minorr(r(r)Úread_file_headerÙs   zMatFile5Reader.read_file_headercCst|ƒ|_t|ƒ|_dS)za Run when beginning read of variables Sets up readers from parameters in `self` N)rr:r;©r<r(r(r)Úinitialize_readäs zMatFile5Reader.initialize_readcCsœ|j ¡\}}|dkstdƒ‚|j ¡|}|tkrbt|j|ƒ}|j |¡|j }|j ¡\}}nd}|j |j¡|t ksˆt d|ƒ‚|j  |¡}||fS)aœ Read header, return header, next position Header has to define at least .name and .is_global Parameters ---------- None Returns ------- header : object object that can be passed to self.read_var_array, and that has attributes .name and .is_global next_position : int position in stream of next variable rzDid not read any bytesFz$Expecting miMATRIX type here, got %d) r:Z read_full_tagÚ ValueErrorr=Útellrr"r;Z set_streamr?rÚ TypeErrorZ read_header)r<ÚmdtypeÚ byte_countZnext_posÚstreamZcheck_stream_limitÚheaderr(r(r)Úread_var_headerðs    zMatFile5Reader.read_var_headercCs|j ||¡S)aµ Read array, given `header` Parameters ---------- header : header object object with fields defining variable header process : {True, False} bool, optional If True, apply recursive post-processing during loading of array. Returns ------- arr : array array with post-processing applied or not according to `process`. )r;Zarray_from_header)r<r^Úprocessr(r(r)Úread_var_arrayszMatFile5Reader.read_var_arrayc Cs„t|tƒr|g}n|dur"t|ƒ}|j d¡| ¡| ¡}g|d<| ¡sl| ¡\}}|j durjdn |j   d¡}||vr’t j d|t dd|d kr¤d }d }nd }|durÆ||vrÆ|j |¡qFz| ||¡}WnFty}z,t j d ||ftddd|}WYd}~n d}~00|j |¡|||<|jrH|d |¡|durF| |¡t|ƒdkrFqlqF|jr|t|ƒS|SdS)z· get variables from stream as dictionary variable_names - optional list of variable names to get If variable_names is None, then get all variables in file NrÚ __globals__ÚNoneÚlatin1zŒDuplicate variable name "%s" in stream - replacing previous with new Consider mio5.varmats_from_mat to split file into single variable filesrD)Ú stacklevelÚÚ__function_workspace__FTz&Unreadable variable "%s", because "%s"zRead error: %s)r#ÚstrÚlistr=rGrWrUÚ end_of_streamr_ÚnameÚdecodeÚwarningsÚwarnrrar ÚWarningÚ is_globalr+ÚremoveÚlenr@r3) r<Zvariable_namesÚmdictrTÚ next_positionrkr`ÚresÚerrr(r(r)Ú get_variables&sZ    ýü ÿý   zMatFile5Reader.get_variablescCs¢|j d¡| ¡| ¡g}| ¡sž| ¡\}}|jdurBdn |j d¡}|dkrZd}|j  |¡}|j rrd}nt   |j d¡}| |||f¡|j |¡q |S) z list variables from stream rNrcrdrfrgÚlogicalÚunknown)r=rGrWrUrjr_rkrlr;Zshape_from_headerÚ is_logicalr ÚgetÚmclassr+)r<ÚvarsrTrtrkÚshapeÚinfor(r(r)Úlist_variables`s    zMatFile5Reader.list_variables) NFFTFTTNF)T)N)Ú__name__Ú __module__Ú __qualname__Ú__doc__rr6rJrUrWr_rarwr€Ú __classcell__r(r(rAr)r4–s&ö&  #  :r4c CsÚt|ƒ}| d¡ttddj}| |¡}| d¡| ¡| ¡| ¡}g}|  ¡sÖ|}|  ¡\}}|j durzdn |j   d¡}| |¡||} | | ¡} t ƒ} |  |¡|  | ¡|  d¡| || f¡qT|S)a Pull variables out of mat 5 file as a sequence of mat file objects This can be useful with a difficult mat file, containing unreadable variables. This routine pulls the variables out in raw form and puts them, unread, back into a file stream for saving or reading. Another use is the pathological case where there is more than one variable of the same name in the file; this routine returns the duplicates, whereas the standard reader will overwrite duplicates in the returned dictionary. The file pointer in `file_obj` will be undefined. File pointers for the returned file-like objects are set at 0. Parameters ---------- file_obj : file-like file object containing mat file Returns ------- named_mats : list list contains tuples of (name, BytesIO) where BytesIO is a file-like object containing mat file contents as for a single variable. The BytesIO contains a string with the original header and a single var. If ``var_file_obj`` is an individual BytesIO instance, then save as a mat file with something like ``open('test.mat', 'wb').write(var_file_obj.read())`` Examples -------- >>> import scipy.io BytesIO is from the ``io`` module in Python 3, and is ``cStringIO`` for Python < 3. >>> mat_fileobj = BytesIO() >>> scipy.io.savemat(mat_fileobj, {'b': np.arange(10), 'a': 'a string'}) >>> varmats = varmats_from_mat(mat_fileobj) >>> sorted([name for name, str_obj in varmats]) ['a', 'b'] rrKrLNrcrd)r4rGrrÚitemsizerHrWrUrYrjr_rkrlrÚwriter+) Zfile_objÚrdrZhdr_lenZraw_hdrrtZ named_matsZstart_positionrTrkr\Zvar_strZout_objr(r(r)Úvarmats_from_matys,)         r‰c@seZdZdZdS)ÚEmptyStructMarkerz= Class to indicate presence of empty matlab struct on output N)rr‚rƒr„r(r(r(r)rŠ¾srŠcCst|tjƒr|S|durdSt|dƒo8t|dƒo8t|dƒ}t|tjƒrHn*|srt|dƒrrtdd„|j ¡Dƒƒ}d}|rÜg}g}| ¡D]:\}}t|tƒr†|d d vr†|  t|ƒt f¡|  |¡q†|rØt  t |ƒg|¡St St |¡}|jjt tjfvr|jd kr||krdS|S) aÙ Convert input object ``source`` to something we can write Parameters ---------- source : object Returns ------- arr : None or ndarray or EmptyStructMarker If `source` cannot be converted to something we can write to a matfile, return None. If `source` is equivalent to an empty dictionary, return ``EmptyStructMarker``. Otherwise return `source` converted to an ndarray with contents for writing to matfile. NÚkeysÚvaluesÚitemsr/css$|]\}}| d¡s||fVqdS)Ú_N)Ú startswith)Ú.0r2Úvaluer(r(r)Ú Ýs  ÿzto_writeable..TrZ _0123456789r()r#r$r%ÚhasattrZgenericÚdictr/rrhr+ÚobjectÚarrayÚtuplerŠZ asanyarrayÚdtypeÚtypeZobject_r~)ÚsourceZ is_mappingr˜rŒÚfieldr‘Únarrr(r(r)Ú to_writeableÂs@ ÿ   ÿ  ÿÿrrKrLZtag_fullZ tag_smalldataZ array_flagsc@sºeZdZdZe de¡Zeed<dd„Z dd„Z dd „Z d,d d „Z d d„Z dd„Zd-dd„Zdd„Zdd„Zdd„Zdd„Zd.dd„Zd d!„Zd"d#„Zd$d%„Zd&d'„Zd(d)„Zd*d+„Zd S)/Ú VarWriter5z% Generic matlab matrix writing class r(r[cCs0|j|_|j|_|j|_|j|_d|_d|_dS)NF)Ú file_streamÚunicode_stringsÚlong_field_namesÚoned_asÚ _var_nameÚ_var_is_global)r<Z file_writerr(r(r)r6s zVarWriter5.__init__cCs|j |jdd¡dS)NÚF©Úorder)rŸr‡Útobytes©r<Úarrr(r(r)Ú write_bytes szVarWriter5.write_bytescCs|j |¡dS©N)rŸr‡)r<Úsr(r(r)Ú write_string szVarWriter5.write_stringNcCsj|durt|jjdd…}|jjtkr4| ¡ ¡}|j|j}|dkrX|  |||¡n|  |||¡dS)z write tag and data Nré) rr˜rhÚ byteorderrÚbyteswapZ newbyteorderr&r†Úwrite_smalldata_elementÚwrite_regular_element)r<rªr[r\r(r(r)Ú write_elements   zVarWriter5.write_elementcCs:t dt¡}|d>||d<|jdd|d<| |¡dS)Nr(éZbyte_count_mdtyper¥r¦Údata)r$ÚzerosÚ NDT_TAG_SMALLr¨r«)r<rªr[r\Útagr(r(r)r²s z"VarWriter5.write_smalldata_elementcCsTt dt¡}||d<||d<| |¡| |¡|d}|rP|j dd|¡dS)Nr(r[r\rOó)r$r·Ú NDT_TAG_FULLr«rŸr‡)r<rªr[r\r¹Zbc_mod_8r(r(r)r³%s   z VarWriter5.write_regular_elementFrc CsÌ|j}|j}|j ¡|_| |j¡t dt ¡}t |d<d|d<|d>|d>B|d>B} || d>B|d<||d <| |¡|  tj |d d ¡t  |¡}|d kr°| |td ¡n |  |t¡d |_d|_dS)a› Write header for given data options shape : sequence array shape mclass - mat5 matrix class is_complex - True if matrix is complex is_logical - True if matrix is logical nzmax - max non zero elements for sparse arrays We get the name and the global flag from the object, and reset them to defaults after we've used them r(Z data_typerOr\érDrZ flags_classÚnzmaxÚi4©r˜rfrFN)r£r¤rŸrYZ _mat_tag_posr«Úmat_tagr$r·ÚNDT_ARRAY_FLAGSrr´r–Zasarrayr²r) r<r~r|Ú is_complexrzr½rkrpÚafÚflagsr(r(r)Ú write_header1s$      zVarWriter5.write_headercCsX|j ¡}|j |¡||d}|dkr2tdƒ‚||jd<| |j¡|j |¡dS)NrOlz-Matrix too large to save with Matlab 5 formatr\)rŸrYrGr rÀr«)r<Ú start_posÚcurr_posr\r(r(r)Úupdate_matrix_tag\s     zVarWriter5.update_matrix_tagcCs||_||_| |¡dS)aˆ Write variable at top level of mat file Parameters ---------- arr : array_like array-like object to create writer for name : str, optional name as it will appear in matlab workspace default is empty string is_global : {False, True}, optional whether variable will be global on load into matlab N)r¤r£r‡)r<rªrkrpr(r(r)Ú write_topgszVarWriter5.write_topcCsø|j ¡}tj |¡r.| |¡| |¡dSt|ƒ}|durRtd|t |ƒfƒ‚t |t ƒrh|  |¡n‚t |t ƒr|tdƒ‚nn|turŽ| ¡n\|jjr¢| |¡nH|jjr¶| |¡n4|jjdvrà|jrÎd}nd}| ||¡n | |¡| |¡dS)z¨ Write `arr` to stream at top and sub levels Parameters ---------- arr : array_like array-like object to create writer for Nz'Could not convert %s (type %s) to arrayzCannot write matlab functions)ÚUÚSÚUTF8Úascii)rŸrYÚscipyÚsparseÚissparseÚ write_sparserÈrrZr™r#rÚ write_objectrr rŠÚwrite_empty_structr˜ÚfieldsÚ write_structZ hasobjectÚ write_cellsÚkindr Ú write_charÚ write_numeric)r<rªZ mat_tag_posrœÚcodecr(r(r)r‡{s6     ÿ         zVarWriter5.writecCs¸|jjdk}|jjdk}zt|jjdd…}Wn@typ|rN| d¡}n|r^| d¡}n | d¡}t}Yn0|jt||j ƒ|||d|rª|  |j ¡|  |j ¡n |  |¡dS)NÚcÚbrZc128Úi1Zf8)rÂrz) r˜r×rrhÚKeyErrorÚastyperrÅrr¢r´ÚrealÚimag)r<rªZimagfZlogifr|r(r(r)rÙ£s(       ý zVarWriter5.write_numericrÍcCsÒ|jdkst |dk¡rJdt |jdg¡}| |t¡| |td¡dSt |ƒ}|j }| |t¡|j j dkrÀ|jrÀt  |¡}tjdt||ƒ|j ¡d}| ¡ |¡}tjt|ƒfd |d}|j|td dS) z5 Write string array `arr` with given `codec` rrf©rrDNrÊr(©r~r˜ÚbufferZS1©r[)r&r$ÚallÚmaxÚndimrÅrr²rr r~r˜r×Úprodr%r ÚTÚcopyrRÚencoderrr´)r<rªrÚr~Zn_charsZst_arrÚstr(r(r)rؼs(    þ þzVarWriter5.write_charcCsž| ¡}| ¡|jjdk}|jjdk}|j}|jt||jƒt|||dkrNdn|d|  |j   d¡¡|  |j   d¡¡|  |j j¡|rš|  |j j¡dS)z Sparse matrices are 2D rÛrÜrr)rÂrzr½r¾N)ZtocscZ sort_indicesr˜r×ZnnzrÅrr¢rr´ÚindicesrßZindptrr¶ràrá)r<rªÚArÂrzZnzr(r(r)rÑçs   ûzVarWriter5.write_sparsecCs<| t||jƒt¡t |¡ d¡}|D]}| |¡q(dS)Nr¥)rÅrr¢rr$Ú atleast_2dÚflattenr‡)r<rªrïÚelr(r(r)rÖûs ÿzVarWriter5.write_cellscCs<| dt¡| tjdtjd¡| tjgtjd¡dS)N)rrrr¿)rÅrr´r$r–Zint32Zint8rVr(r(r)rÓs zVarWriter5.write_empty_structcCs"| t||jƒt¡| |¡dSr¬)rÅrr¢rÚ _write_itemsr©r(r(r)rÕ sÿzVarWriter5.write_structcCs¶dd„|jjDƒ}tdd„|Dƒƒd}|jr2dp4d}||krNtd|dƒ‚| tj|gdd ¡|jtj|d |d td t  |¡  d ¡}|D]}|D]}|  ||¡qœq”dS) NcSsg|] }|d‘qSrâr()rr1r(r(r)Ú óz+VarWriter5._write_items..cSsg|] }t|ƒ‘qSr()rr)rÚ fieldnamer(r(r)rôrõré@é z+Field names are restricted to %d charactersr¾r¿zS%drår¥) r˜Údescrrçr¡rXr´r$r–rrðrñr‡)r<rªÚ fieldnamesÚlengthÚ max_lengthrïròr1r(r(r)rós ÿþzVarWriter5._write_itemscCs<| t||jƒt¡|jtj|jddtd|  |¡dS)zmSame as writing structs, except different mx class, and extra classname element after header rËr¿råN) rÅrr¢rr´r$r–Ú classnamerrór©r(r(r)rÒ sÿÿzVarWriter5.write_object)N)FFr)rÍ)rr‚rƒr„r$r·r»rÀrr6r«r®r´r²r³rÅrÈrÉr‡rÙrØrÑrÖrÓrÕrórÒr(r(r(r)ržüs0   û + ( +ržc@s0eZdZdZed dd„ƒZdd„Zd d d „ZdS) ÚMatFile5Writerz Class for writing mat5 files FNÚrowcCs:||_||_||_|r||_ng|_||_||_d|_dS)a< Initialize writer for matlab 5 format files Parameters ---------- %(do_compression)s %(unicode_strings)s global_vars : None or sequence of strings, optional Names of variables to be marked as global for matlab %(long_fields)s %(oned_as)s N)rŸÚdo_compressionr Ú global_varsr¡r¢Ú_matrix_writer)r<rŸrr rr¡r¢r(r(r)r6.szMatFile5Writer.__init__cCsXt dt¡}dtjt ¡f|d<d|d<tjddt d¡d|d <|j   |  ¡¡dS) Nr(z0MATLAB 5.0 MAT-file Platform: %s, Created on: %srMérNZS2iIMrãZ endian_test) r$r·Ú NDT_FILE_HDRÚosrkÚtimeÚasctimer%Zuint16rŸr‡r¨)r<rTr(r(r)Úwrite_file_headerKs  ÿþ z MatFile5Writer.write_file_headerc Csæ|dur|j ¡dk}|r"| ¡t|ƒ|_| ¡D]¬\}}|ddkrJq4||jv}|jrÊtƒ}||j_|j  ||  d¡|¡t   |  ¡¡}t dt¡}t|d<t|ƒ|d<|j | ¡¡|j |¡q4|j  ||  d¡|¡q4dS)aò Write variables in `mdict` to stream Parameters ---------- mdict : mapping mapping with method ``items`` returns name, contents pairs where ``name`` which will appear in the matlab workspace in file load, and ``contents`` is something writeable to a matlab file, such as a NumPy array. write_header : {None, True, False}, optional If True, then write the matlab file header before writing the variables. If None (the default) then write the file header if we are at position 0 in the stream. By setting False here, and setting the stream position to the end of the file, you can append variables to a matlab file NrrŽrdr(r[r\)rŸrYrržrrrrrrÉrìÚzlibÚcompressÚgetvaluer$Úemptyr»rrrr‡r¨) r<rsrÅrkÚvarrpr]Zout_strr¹r(r(r)Ú put_variablesVs(     zMatFile5Writer.put_variables)FFNFrÿ)N)rr‚rƒr„rr6rrr(r(r(r)rþ+sû  rþ)>r„rrr7r ÚiorrmZnumpyr$Z scipy.sparserÎZbyteordercodesrrZmiobaserrrr r r r r rZ mio5_utilsrZ mio5_paramsrrrrrrrrrrrrrrrrr r!Ústreamsr"r*r-r,r3r4r‰rŠrrr»r¸rÁržrþr(r(r(r)Ús<I , P  dE41