U ˜­“]•ã@s¢dZddlZddlZddlZddlZddlZddlZddlZddlm Z m Z m Z ddl m Z ddlmZddlmZe e¡ZGdd„deƒZd d „Zd d „ZdS) z@ApacheParser is a member object of the ApacheConfigurator class.éN)ÚDictÚListÚSet)Úerrors)Úos)Ú constantsc@sŽeZdZdZe d¡ZedddddgƒZdbd d „Z d d „Z dd„Z dd„Z dd„Z dd„Zdd„Zdd„Zdd„Zdd„Zdd„Zd d!„Zd"d#„Zd$d%„Zd&d'„Zd(d)„Zd*d+„Zd,d-„Zd.d/„Zd0d1„Zdcd3d4„Zddd5d6„Zd7d8„Zd9d:„Z d;d<„Z!ded=d>„Z"dfd@dA„Z#dBdC„Z$dDdE„Z%dFdG„Z&dHdI„Z'dJdK„Z(dLdM„Z)dNdO„Z*dPdQ„Z+dRdS„Z,dTdU„Z-dVdW„Z.dXdY„Z/dZd[„Z0d\d]„Z1d^d_„Z2d`da„Z3dS)gÚ ApacheParseraÉClass handles the fine details of parsing the Apache Configuration. .. todo:: Make parsing general... remove sites-available etc... :ivar str root: Normalized absolute path to the server root directory. Without trailing slash. :ivar set modules: All module names that are currently enabled. :ivar dict loc: Location to place directives, root - configuration origin, default - user config file, name - NameVirtualHost, z \$\{[^ \}]*}Ú*ú?ú\ú[ú]N©éécCsî||_d|_| ¡| ¡s&t d¡‚tƒ|_i|_i|_ t j   |¡|_ d| ¡i|_| |jd¡|dkrv| ¡| ¡| ¡|j | ¡¡t |j¡|_|rÊ| t j   |¡d|j d¡¡|dkrê|jdddrêt d ¡‚dS) NzˆApache plugin support requires libaugeas0 and augeas-lenses version 1.2.0 or higher, please make sure you have you have those installed.Úrootrú/Z vhost_filesZDefineF)ÚexcludezError parsing runtime variables)Ú configuratorÚaugÚ init_augeasÚcheck_aug_versionrZNotSupportedErrorÚsetÚmodulesÚ parser_pathsÚ variablesrÚpathÚabspathrÚ_find_config_rootÚlocÚ parse_fileÚupdate_runtime_variablesÚstandardize_exclÚ parse_modulesÚupdateÚ_set_locationsÚcopyÚdeepcopyÚexisting_pathsÚoptionÚfind_dirÚ PluginError)ÚselfrZ vhostrootÚversionr©r.ú7/usr/lib/python3/dist-packages/certbot_apache/parser.pyÚ__init__%s4ÿ ÿzApacheParser.__init__cCsVz ddl}Wntk r*t d¡‚YnX|jtj|jj|jjB|jj Bd|_ dS)z' Initialize the actual Augeas instance rNzProblem in Augeas installation)ZloadpathÚflags) ÚaugeasÚ ImportErrorrÚNoInstallationErrorZAugeasrZAUGEAS_LENS_DIRZNONEZNO_MODL_AUTOLOADZ ENABLE_SPANr)r,r2r.r.r/r[s ÿþûzApacheParser.init_augeasc Csv|j d¡}|D]`}|j |d¡}|r||krd |dt|ƒd…|j |d¡|j |d¡¡}t |¡‚qdS) zÜVerify Augeas can parse all of the lens files. :param str lens: lens to check for errors :raises .errors.PluginError: If there has been an error in parsing with the specified lens. ú/augeas//errorz/lensz@There has been an error in parsing the file {0} on line {1}: {2}é éz/linez/messageN)rÚmatchÚgetÚformatÚlenrr+)r,ZlensZ error_filesrZ lens_pathÚmsgr.r.r/Úcheck_parsing_errorsls  ûÿz!ApacheParser.check_parsing_errorscCsR|j dd¡z|j d¡}Wn"tk r@|j d¡YdSX|j d¡|S)z Checks that we have recent enough version of libaugeas. If augeas version is recent enough, it will support case insensitive regexp matchingz/test/path/testing/argZaRgUMeNTz,/test//*[self::arg=~regexp('argument', 'i')]z /test/pathF)rrr8Ú RuntimeErrorÚremove)r,Úmatchesr.r.r/r†sÿ  zApacheParser.check_aug_versionc Cs¶|j d¡}|j dd¡|j d¡}z|j ¡Wn4ttfk rh| |¡d|j_ t   d¡‚YnX|j d|¡|j d¡}tƒ}|r²|D]}|  |j |¡dd…¡q’|S) a™Lists files that have modified Augeas DOM but the changes have not been written to the filesystem yet, used by `self.save()` and ApacheConfigurator to check the file state. :raises .errors.PluginError: If there was an error in Augeas, in an attempt to save the configuration, or an error creating a checkpoint :returns: `set` of unsaved files z /augeas/saveZnoopr5Úz-Error saving files, check logs for more info.z/augeas/events/savedr7N) rr9rr8Úsaver>ÚIOErrorÚ_log_save_errorsrÚ save_notesrr+Úadd)r,Z save_stateÚex_errsZ save_pathsÚ save_filesrr.r.r/Ú unsaved_files•s$   ÿ  zApacheParser.unsaved_filescCs&| ¡r"|jjd7_|j ¡dS)zÙMakes sure that all Augeas dom changes are written to files to avoid loss of configuration directives when doing additional augeas parsing, causing a possible augeas.load() resulting dom reset z (autosave)N)rIrrErB©r,r.r.r/Úensure_augeas_state¼sz ApacheParser.ensure_augeas_statecCs>d|j_|j ¡|r:|D]}|j d|¡q|j ¡dS)zõSaves all changes to the configuration files. save() is called from ApacheConfigurator to handle the parser specific tasks of saving. :param list save_files: list of strings of file paths that we need to save. rAz/files/N)rrErrBr?Úload)r,rHZsfr.r.r/rBÆs  zApacheParser.savecs6|j d¡}t dd ‡fdd„|Dƒ¡|jj¡dS)zfLog errors due to bad Augeas save. :param list ex_errs: Existing errors before save r5z2Unable to save files: %s. Attempted Save Notes: %sz, c3s*|]"}|ˆkr|dt|ƒd…VqdS)r6r7N)r;)Ú.0Úerr©rGr.r/Ú ãsþz0ApacheParser._log_save_errors..N)rr8ÚloggerÚerrorÚjoinrrE)r,rGZnew_errsr.rOr/rDÙs  ýzApacheParser._log_save_errorscCsd| tdƒ|¡s`t d|t|ƒ¡| t|ƒd|¡tj |¡}tj  |¡}|j   |g¡  |¡dS)zÃAdd Include for a new configuration file if one does not exist :param str main_config: file path to main Apache config file :param str inc_path: path of file to include ÚIncludezAdding Include %s to %sN) r*Úcase_irQÚdebugÚ get_aug_pathÚadd_dirrrÚdirnameÚbasenamer(Ú setdefaultÚappend)r,Z main_configZinc_pathZnew_dirZnew_filer.r.r/Ú add_includeçsÿþ  zApacheParser.add_includecCsH|d|jkr|j |d¡d|d|jkrD|j d|d¡dS)z%Shortcut for updating parser modules.Ú_moduleZmod_z.cN)rrF)r,Úmod_namer.r.r/Úadd_modúszApacheParser.add_modcCstƒ|_| ¡| ¡dS)zgReset the loaded modules list. This is called from cleanup to clear temporarily loaded modules.N)rrÚupdate_modulesr#rJr.r.r/Ú reset_modulesszApacheParser.reset_modulesc Cs´tƒ}| d¡}t|ƒ}d}t|ƒ|kr¤t|ƒ}tj ||¡D]b\}}| |¡}| |¡}|rŒ|rŒ| |¡| t j   |¡dd…d¡q>t   d|dd…¡q>q|j |¡dS)zãIterates on the configuration until no new modules are loaded. ..todo:: This should be attempted to be done with a binary to avoid the iteration issue. Else... parse and enable mods at same time. Z LoadModuleéÿÿÿÿNéþÿÿÿÚcz8Could not read LoadModule directive from Augeas path: %sr7)rr*Úiterr;ÚsixZmovesÚzipÚget_argrFrrrZrQrVrr$) r,Zmodsr@ÚiteratorZ prev_sizeZ match_nameZmatch_filenamer_Z mod_filenamer.r.r/r#s&  ÿ      ÿzApacheParser.parse_modulescCs| ¡| ¡| ¡dS)zAUpdate Includes, Defines and Includes from httpd config dump dataN)Úupdate_definesÚupdate_includesrarJr.r.r/r!$sz%ApacheParser.update_runtime_variablescCsœtƒ}|j d¡dddg}| |d¡}z| d¡Wntk rJYdSX|D]@}| d¡dkrvt d ¡t   d ¡‚|  d¡}|d ||d <qP||_ dS) zGet Defines from httpd processÚctlú-tú-DZ DUMP_RUN_CFGzDefine: ([^ \n]*)Nú=éz8Unexpected number of equal signs in runtime config dump.z&Error parsing Apache runtime variablesrr) Údictrr)Úparse_from_subprocessr?Ú ValueErrorÚcountrQrRrr+Ú partitionr)r,rZ define_cmdr@r8Úpartsr.r.r/rk*s$ÿ  ÿ zApacheParser.update_definescCsP| d¡}|j d¡dddg}| |d¡}|rL|D]}| |¡s2| |¡q2dS)z>Get includes from httpd process, and add them to DOM if neededrTrmrnroZ DUMP_INCLUDESz \(.*\) (.*)N)r*rr)rsÚparsed_in_currentr )r,Ú_Zinc_cmdr@Úir.r.r/rlAs ÿ  zApacheParser.update_includescCs<|j d¡dddg}| |d¡}|D]}| | ¡¡q$dS)z:Get loaded modules from httpd process, and add them to DOMrmrnroZ DUMP_MODULESz (.*)_moduleN)rr)rsr`Ústrip)r,Zmod_cmdr@Úmodr.r.r/raQs ÿ zApacheParser.update_modulescCs| |¡}t |¡ |¡S)zÙGet values from stdout of subprocess command :param list command: Command to run :param str regexp: Regexp for parsing :returns: list parsed from command output :rtype: list )Ú_get_runtime_cfgÚreÚcompileÚfindall)r,ÚcommandZregexpÚstdoutr.r.r/rsZs z"ApacheParser.parse_from_subprocessc Cs„z&tj|tjtjdd}| ¡\}}Wn8ttfk r^t d|tj ¡t   d  |¡¡‚YnX|j dkr€t d|¡t   d¡‚|S)zwGet runtime configuration info. :param command: Command to run :returns: stdout from command T)r‚ÚstderrZuniversal_newlinesz2Error running command %s for runtime parameters!%sz-Error accessing loaded Apache parameters: {0}rz$Error in checking parameter list: %sz^Apache is unable to check whether or not the module is loaded because Apache is misconfigured.)Ú subprocessÚPopenÚPIPEZ communicateÚOSErrorrtrQrRrÚlineseprZMisconfigurationErrorr:Ú returncodeZwarning)r,rÚprocr‚rƒr.r.r/r}gs2üþÿÿ   ÿzApacheParser._get_runtime_cfgcCs®g}|dkr@t|ƒD](\}}| d¡r| ||dd…¡qnjt|ƒD]`\}}| d|¡rH|t|ƒdksˆ||d d|d¡sH| ||dtd|ƒ …¡qH|S)aFilter out directives with specific number of arguments. This function makes the assumption that all related arguments are given in order. Thus /files/apache/directive[5]/arg[2] must come immediately after /files/apache/directive[5]/arg[1]. Runs in 1 linear pass. :param string matches: Matches of all directives with arg nodes :param int args: Number of args you would like to filter :returns: List of directives that contain # of arguments. (arg is stripped off) rqú/argNéüÿÿÿú/arg[%d])Ú enumerateÚendswithr\r;)r,r@ÚargsZfilteredrzr8r.r.r/Úfilter_args_num†s ÿÿ"zApacheParser.filter_args_numcCsŠ| |d¡}|j |ddd¡|d}|j ||¡t|ƒdkrZ|j |d|d¡n,t|ƒD]"\}}|j d ||df|¡qbd S) aáAdds directive and value to IfMod ssl block. Adds given directive and value along configuration path within an IfMod mod_ssl.c block. If the IfMod block does not exist in the file, it is created. :param str aug_conf_path: Desired Augeas config path to add directive :param str directive: Directive you would like to add, e.g. Listen :param args: Values of the directive; str "443" or list of str :type args: list z mod_ssl.cÚargÚ directiveFz directive[1]rqr‹rz %s/arg[%d]N)Ú get_ifmodrÚinsertrr;rŽ)r,Ú aug_conf_pathr“rZ if_mod_pathZnvh_pathrzr’r.r.r/Úadd_dir_to_ifmodssl¥s  z ApacheParser.add_dir_to_ifmodsslFcCs8|j d||f¡}|s&| |||¡S|d d¡dS)aReturns the path to and creates one if it doesn't exist. :param str aug_conf_path: Augeas configuration path :param str mod: module ie. mod_ssl.c :param bool beginning: If the IfModule should be created to the beginning of augeas path DOM tree. :returns: Augeas path of the requested IfModule directive that pre-existed or was created during the process. The path may be dynamic, i.e. .../IfModule[last()] :rtype: str z%s/IfModule/*[self::arg='%s']rr’)rr8Ú create_ifmodÚ rpartition)r,r–r|Ú beginningZif_modsr.r.r/r”¿s ÿzApacheParser.get_ifmodcCsn|r0d |¡}|j d |¡dd¡d |¡}n,d |¡}d |¡}|j |d¡d  |¡}|j ||¡|S) aµCreates a new and returns its path. :param str aug_conf_path: Augeas configuration path :param str mod: module ie. mod_ssl.c :param bool beginning: If the IfModule should be created to the beginning of augeas path DOM tree. :returns: Augeas path of the newly created IfModule directive. The path may be dynamic, i.e. .../IfModule[last()] :rtype: str z{}/IfModule[1]/argz{}/directive[1]ZIfModuleTz{}/IfModule[1]/z{}/IfModule[last() + 1]z{}/IfModule[last()]/argrAz{}/IfModule[last()]/)r:rr•r)r,r–r|ršZ c_path_argZretpathZc_pathr.r.r/r˜Õs  ÿ    zApacheParser.create_ifmodcCs^|j |d|¡t|tƒrHt|dƒD]\}}|j d||f|¡q&n|j |d|¡dS)aAppends directive to the end fo the file given by aug_conf_path. .. note:: Not added to AugeasConfigurator because it may depend on the lens :param str aug_conf_path: Augeas configuration path to add directive :param str directive: Directive to add :param args: Value of the directive. ie. Listen 443, 443 is arg :type args: list or str z/directive[last() + 1]rqz%s/directive[last()]/arg[%d]z/directive[last()]/argN)rrÚ isinstanceÚlistrŽ)r,r–r“rrzÚvaluer.r.r/rXðs   ÿzApacheParser.add_dircCsr|d}|j |dd¡|j ||¡t|tƒr\t|dƒD]\}}|j |d||¡q:n|j |d|¡dS)a)Adds the directive to the beginning of defined aug_conf_path. :param str aug_conf_path: Augeas configuration path to add directive :param str dirname: Directive to add :param args: Value of the directive. ie. Listen 443, 443 is arg :type args: list or str z /directive[1]r“Trqrr‹N)rr•rr›rœrŽ)r,r–rYrZ first_dirrzrr.r.r/Úadd_dir_beginnings zApacheParser.add_dir_beginningcCs|j |d|¡dS)z§Adds the comment to the augeas path :param str aug_conf_path: Augeas configuration path to add directive :param str comment: Comment content z/#comment[last() + 1]N)rr)r,r–Úcommentr.r.r/Ú add_commentszApacheParser.add_commentcCsR|st|jƒ}|j d|¡}g}|D]&}|j |¡}|r&||kr&| |¡q&|S)aFinds a comment with specified content from the provided DOM path :param str arg: Comment content to search :param str start: Beginning Augeas path to begin looking :returns: List of augeas paths containing the comment content :rtype: list z%s//*[label() = '#comment'])rWrrr8r9r\)r,r’ÚstartZcommentsZresultsrŸZ c_contentr.r.r/Ú find_commentss     zApacheParser.find_commentsTc Csâ|st|jdƒ}dt|ƒtdƒtdƒf}|j d||f¡}|rN| |¡}|dkr\d}n dt|ƒ}g}|D]l} |j | ¡ ¡} | d ks”| d krº| |  |||  |  | d¡¡|¡¡| | ¡krp| |j | |¡¡qp|S) a Finds directive in the configuration. Recursively searches through config files to find directives Directives should be in the form of a case insensitive regex currently .. todo:: arg should probably be a list .. todo:: arg search currently only supports direct matching. It does not handle the case of variables or quoted arguments. This should be adapted to use a generic search for the directive and then do a case-insensitive self.get_arg filter Note: Augeas is inherently case sensitive while Apache is case insensitive. Augeas 1.0 allows case insensitive regexes like regexp(/Listen/, "i"), however the version currently supported by Ubuntu 0.10 does not. Thus I have included my own case insensitive transformation by calling case_i() on everything to maintain compatibility. :param str directive: Directive to look for :param arg: Specific value directive must have, None if all should be considered :type arg: str or None :param str start: Beginning Augeas path to begin looking :param bool exclude: Whether or not to exclude directives based on variables and enabled modules rz(%s)|(%s)|(%s)rTZIncludeOptionalz$%s//*[self::directive=~regexp('%s')]Nr‹z/*[self::arg=~regexp('%s')]ZincludeZincludeoptional) rWrrUrr8Ú _exclude_dirsr9ÚlowerÚextendr*Ú_get_include_pathri) r,r“r’r¡rZregexr@Z arg_suffixZordered_matchesr8Zdir_r.r.r/r*4s6þ ÿ  ý zApacheParser.find_dircs6|ddkr|d}ˆj |d¡}‡fdd„|DƒS)zÑ Tries to fetch all arguments for a directive. See get_arg. Note that if match is an ancestor node, it returns all names of child directives as well as the list of arguments. rcrr csg|]}ˆ |¡‘qSr.)ri)rMr’rJr.r/Ú Œsz-ApacheParser.get_all_args..)rr8)r,r8Zallargsr.rJr/Ú get_all_args€s zApacheParser.get_all_argsc Csz|j |¡}|sdS| d¡}tj |¡}|D]F}z| ||j|dd…¡}Wq.tk rrt   d|¡‚Yq.Xq.|S)zŽUses augeas.get to get argument value and interprets result. This also converts all variables and parameters appropriately. Nú'"rrczError Parsing variable: %s) rr9r{rÚarg_var_interpreterr€ÚreplacerÚKeyErrorrr+)r,r8rrÚvarr.r.r/riŽs   zApacheParser.get_argcCsJd|jfd|jfg}g}|D](}|D]}| ||¡s$qq$| |¡q|S)z>Exclude directives that are not loaded into the configuration.ZifmoduleZifdefine)rrÚ _pass_filterr\)r,r@ÚfiltersZ valid_matchesr8Úfilter_r.r.r/r£«s  zApacheParser._exclude_dirscCsŒ| ¡}| |d¡}|dkrˆ| d|¡}|j |d|…d¡}| d¡rf|dd…|dkrvdSn||dkrvdS| |d|¡}qd S) zÐDetermine if directive passes a filter. :param str match: Augeas path :param list filter: list of tuples of form [("lowercase if directive", set of relevant parameters)] rrcrNr‹ú!rqFT)r¤Úfindrr9Ú startswith)r,r8r°Zmatch_lZlast_match_idxZ end_of_ifZ expressionr.r.r/r®¹s   zApacheParser._pass_filtercCs¸| d¡}| d¡s.tj tj |j|¡¡}n tj |¡}tj |¡r\| tj |d¡¡n | |¡|  d¡}t |ƒD],\}}t dd„|Dƒƒrxd|  |¡||<qxd |¡}t |ƒS)a@Converts an Apache Include directive into Augeas path. Converts an Apache Include directive argument into an Augeas searchable path .. todo:: convert to use os.path.join() :param str arg: Argument of Include directive :returns: Augeas path string :rtype: str r©rr css|]}|tjkVqdS)N)rÚ fnmatch_chars)rMÚcharr.r.r/rPsz1ApacheParser._get_include_path..z* [label()=~regexp('%s')])r{r³rrÚnormpathrSrÚisdirr ÚsplitrŽÚanyÚ fnmatch_to_rerW)r,r’Z split_argÚidxr¸r.r.r/r¦Ös      ÿ  zApacheParser._get_include_pathcCs.tjdkrt |¡dd…St |¡dd…S)aäMethod converts Apache's basic fnmatch to regular expression. Assumption - Configs are assumed to be well-formed and only writable by privileged users. https://apr.apache.org/docs/apr/2.0/apr__fnmatch_8h_source.html http://apache2.sourcearchive.com/documentation/2.2.16-6/apr__fnmatch_8h_source.html :param str clean_fn_match: Apache style filename match, like globs :returns: regex suitable for augeas :rtype: str )ér7Niùÿÿÿréýÿÿÿ)ÚsysÚ version_infoÚfnmatchÚ translate)r,Zclean_fn_matchr.r.r/rº s zApacheParser.fnmatch_to_recCsT| |¡\}}| ¡|rP|j d|¡}|sP|r<| |¡| |¡|j ¡dS)zÙParse file with Augeas Checks to see if file_path is parsed by Augeas If filepath isn't parsed, the file is added and Augeas is reloaded :param str filepath: Apache config file path z&/augeas/load/Httpd['%s' =~ glob(incl)]N)Ú_check_path_actionsrKrr8Ú_remove_httpd_transformÚ_add_httpd_transformrL)r,ÚfilepathÚuse_newÚ remove_oldZinc_testr.r.r/r s ÿ  zApacheParser.parse_filecCs| ||j¡S)a<Checks if the file path is parsed by current Augeas parser config ie. returns True if the file is found on a path that's found in live Augeas configuration. :param str filep: Path to match :returns: True if file is parsed in existing configuration tree :rtype: bool )Ú_parsed_by_parser_pathsr©r,Úfilepr.r.r/rx;s zApacheParser.parsed_in_currentcCs| ||j¡S)a[Checks if the file path is parsed by existing Apache config. ie. returns True if the file is found on a path that matches Include or IncludeOptional statement in the Apache configuration. :param str filep: Path to match :returns: True if file is parsed in existing configuration tree :rtype: bool )rÈr(rÉr.r.r/Úparsed_in_originalGs zApacheParser.parsed_in_originalc Cs>| ¡D]0}||D]"}t |tj ||¡¡rdSqqdS)znHelper function that searches through provided paths and returns True if file path is found in the setTF)ÚkeysrÀrrrS)r,rÊÚpathsZ directoryÚfilenamer.r.r/rÈSs    z$ApacheParser._parsed_by_parser_pathscCsbz.)rSr~Úescape)Ústringr.r.r/rUÜs ÿrUcCsd|S)zTReturn augeas path for full filepath. :param str file_path: Full filepath z/files%sr.)Z file_pathr.r.r/rWìsrW)rÙr&rÀZloggingr~r„r¾rgZacme.magic_typingrrrZcertbotrZcertbot.compatrZcertbot_apacherZ getLoggerrÖrQÚobjectrrUrWr.r.r.r/Ús,    N